Looking for language and framework for data munging/wrangling I have an extensive series of projects that require importing data from many different types of different sources (websites, web apis, sensors, legacy text files, etc). Is there a good framework (preferably open source) that has been built bottom up to tackle this type of problem? Preferably one leveraging an appropriate language (python, most likely) and already having an extensive plugin database.
If you are interested in a very high-level (enterprise architecture) framework, I suggest you to take a look at the MIKE2.0 Methodology. Being an information management framework, MIKE2.0 has, certainly, much wider coverage than the domain of your interest, but it is a solid, interesting and open (licensed under the Creative Commons Attribution License) framework. A better fit for your focus is the Extract, transform, load (ETL) framework, which is extremely popular in contexts of Business Intelligence and Data Warehousing. On a more practical note, you might want to check my answer on Quora on open source master data management (MDM) solutions. Pay attention to the Talend solutions (disclaimer: I am not affiliated with this or any company), which cover a wide spectrum of MDM, ETL and data integration domains as open source and commercial offerings.