What is the most reliable and fault tolerant way to integrate 3rd-party data structures via a web service in Drupal 7? I've seen a number of strategies for integrating remote data structures in Drupal. The strategies have seemed to evolve as certain modules have stabilized and use cases have been tried.

Imagine we have a data structure "Farmers Markets" that is represented by a number of data types (market, market_hours, vender, stall, produce) etc that are exposed via a REST API. The IDs for the external service would need to relate in Drupal, ie when loading a "market" we would want to grab data from the 'market_hours' and 'stall'. What would be the best way to represent that as read-only content in Drupal that is synced on a regular basis?

I'm trying to evaluate this with the following criteria:

Data Structures in Drupal:

Nodes vs Custom Entities

A number of scenarios involving web services I've seen use custom entities. It simplifies the CRUD ops. However, these items would be "content" in that they would be viewed publicly.

Storage (Local vs Remote):

I've seen a couple of examples where services are loaded as remote entities, which this module creates a library for: https://drupal.org/project/wsdata . This sounds most appealing, but haven't seen a lot of use cases. There also examples of custom code: https://drupal.org/sandbox/fago/1493180

Syncing Data:

Feeds vs Migrate vs Guzzle vs 'Web Service Client' vs 'Web Services Data'

There are a number of options. Feeds now supports entities. Migrate seems a lot cleaner than feeds, especially for custom scenarios. I've also seen folks using a guzzle client to grab sync with remote services: http://drupalcode.org/project/ckan.git/blob/refs/heads/ckan_dgu_7.x-1.x:/ckan.drush.inc#l273. I've also noticed the WS Client module https://drupal.org/project/wsclient provides an option that has been created specifically as a rest client. Web Services Data loads directly from a service and caches it locally.

Thanks for any thoughts.
1. Reformulating the question

Your example suggests that the data is read only on the Drupal side, with one-way syncing only. I think this is the most important factor to consider here, because in effect whatever solution you implement will be a variant of remote storage, syncing and local caching - even if the local caching ends up being entities in the Drupal database.

So the question, rather than being "local storage vs remote storage" is going to be:


Should you cache the data locally at all ;
Should you cache the data as actual entities, and use Feeds (or similar) to synchronise the data regularly ; OR
Should you use some custom made module that provides the syncing and the caching.


An article you might be interested is in "Remote Entities in Drupal 7".

2. Caching the data

In general, caching the data is a good idea:


You are protected against outages of the other services, or timeouts
in the connection ;
Having your data present in your Drupal database will speed up operations ;
Having your data present in your Drupal database will mean that you are more likely to get integration with other modules, such as Views (though this is not guaranteed).


The only advantage of not caching the data is that you never get stale data, which in some cases is preferable - sometimes it is preferable to display no data rather than stale data. I do not see this as a benefit in the example you have given, so I will focus this answer on a solution that involves local caching.

3. Local entities + Syncing

If you go for the option of having local entities and syncing them yourself, then we come back to your original questions:


Should you use nodes or custom entities ;
Which module is best for syncing.


3.1 Nodes vs Custom entities


The definition of what exactly is a node is one that is quite open. The documentation page on nodes suggests that nodes are "posting" that are "stored" on your site - neither of which apply to your data ;
As a Drupal developer I would expect that if something is a node then I should be able to manipulate it on the site itself ;
As Drupal user, I would similarly expect that nodes can be edited ;
This Drupal 8 issue https://drupal.org/node/2019031 suggests that the concept of "read only" is one that would apply at the entity level, rather than the bundle level. If this ever gets implemented you would benefit from it by having gone down this route.


To summarise: your data being read only and stored remotely it makes more sense to use a custom entity type to represent your data.

3.2 Syncing

For the second part, the two main modules for this are, as you suggest, Feeds and Migrate.

The difference between Feeds and Migrate is that Feeds is build for regular importing of content, while Migrate is build for one-off porting of content from one place to another. Migrate does support updating existing data, however given that both modules are well supported it makes more sense to use the module that was build for the task at hand - Feeds is a better match.

Having used both modules myself (Feeds for syncing, Migrating for migrating) I do not find Feeds to be more messy than Migrate. Migrate has required more custom code in my experience, though migrating entire sites is more complex than importing single content types, so it's hard to compare.

4. Custom module for remote storage, syncing + caching

There are a number of modules out there that can help with this task.

You mentioned the Web Services Data module, and others have mentioned the Data module. Another option to consider is the Remote Entity API module. Note that the only one of those I have experience with is the Data module.


The Web Services Data module does not yet have a release - which may indicate the code is not yet stable, the API might change and so on. It does not support Entity Field Queries (according to it's project page), and a quick browse of the code repository shows no evidence of it having Views support - so you would not be able to use Views to display your entities ;
The Data module is more oriented, in my experience, towards non-developers who have data in a table and want to expose it to views. I have found the Drupal 6 version to be quite frustrating to use - though that might have changed since then ;
The Remote Entity API module sounds quite promising - it supports fetching and caching of remote entities, and has Views support. It's only on alpha release - so API might still change. At first glance it does not seem to have Entity Field Query support either, and it only support one type of remote service so you would have to implement your own.


Conclusion

Given that none of the remote storage modules support Entity Field Queries, using actual entities + feeds is the solution that will give you the best integration with your Drupal site.

If Views support is enough, and you are not worried about potential integration with other modules via Entity Field Queries, then using the Remote Entity API might be the way to go - you would need to implement your own remote interface however.