text:
MAD Skills:New Analysis Practices for Big Data
Jeffrey Cohen
Brian Dolan
Fox Audience Network
Mark Dunlap
Evergreen Technologies
Joseph M.Hellerstein
Caleb Welton
As massive data acquisition and storage becomes increas-
ingly aordable,a wide variety of enterprises are employing
statisticians to engage in sophisticated data analysis.In this
paper we highlight the emerging practice of Magnetic,Ag-
ile,Deep (MAD) data analysis as a radical departure from
traditional Enterprise Data Warehouses and Business Intel-
ligence.We present our design philosophy,techniques and
experience providing MAD analytics for one of the world's
largest advertising networks at Fox Audience Network,us-
ing the Greenplum parallel database system.We describe
database design methodologies that support the agile work-
ing style of analysts in these settings.We present data-
parallel algorithms for sophisticated statistical techniques,
with a focus on density methods.Finally,we re ect on
database system features that enable agile design and exi-
ble algorithmdevelopment using both SQL and MapReduce
interfaces over a variety of storage mechanisms.
If you are looking for a career where your services will be
in high demand,you should nd something where you provide
a scarce,complementary service to something that is getting
ubiquitous and cheap.So what's getting ubiquitous and cheap?
Data.And what is complementary to data?Analysis.
{ Prof.Hal Varian,UCBerkeley,Chief Economist at Google [5]
mad (adj.):an adjective used to enhance a noun.
1- dude,you got skills.
2- dude,you got mad skills.
{ UrbanDictionary.com [12]
Standard business practices for large-scale data analysis cen-
ter on the notion of an\Enterprise Data Warehouse"(EDW)
that is queried by\Business Intelligence"(BI) software.BI
tools produce reports and interactive interfaces that summa-
rize data via basic aggregation functions (e.g.,counts and
averages) over various hierarchical breakdowns of the data
Permission to copy without fee all or part of this material is granted provided
that the copies are not made or distributed for direct commercial advantage,
the VLDBcopyright notice andthe title of the publicationandits date appear,
and notice is given that copying is by permission of the Very Large Data
Base Endowment.To copy otherwise,or to republish,to post on servers
or to redistribute to lists,requires a fee and/or special permission from the
VLDB '09,August 24-28,2009,Lyon,France
Copyright 2009 VLDB Endowment,ACM000-0-00000-000-0/00/00.
into groups.This was the topic of signicant academic re-
search and industrial development throughout the 1990's.
Traditionally,a carefully designed EDW is considered to
have a central role in good IT practice.The design and
evolution of a comprehensive EDW schema serves as the
rallying point for disciplined data integration within a large
enterprise,rationalizing the outputs and representations of
all business processes.The resulting database serves as the
repository of record for critical business functions.In addi-
tion,the database server storing the EDWhas traditionally
been a major computational asset,serving as the central,
scalable engine for key enterprise analytics.The concep-
tual and computational centrality of the EDW makes it a
mission-critical,expensive resource,used for serving data-
intensive reports targeted at executive decision-makers.It is
traditionally controlled by a dedicated IT sta that not only
maintains the system,but jealously controls access to ensure
that executives can rely on a high quality of service.[13]
While this orthodox EDW approach continues today in
many settings,a number of factors are pushing towards a
very dierent philosophy for large-scale data management in
the enterprise.First,storage is now so cheap that small sub-
groups within an enterprise can develop an isolated database
of astonishing scale within their discretionary budget.The
world's largest data warehouse from just over a decade ago
can be stored on less than 20 commodity disks priced at
under $100 today.A department can pay for 1-2 orders
of magnitude more storage than that without coordinating
with management.Meanwhile,the number of massive-scale
data sources in an enterprise has grown remarkably:mas-
sive databases arise today even fromsingle sources like click-
streams,software logs,email and discussion forum archives,
etc.Finally,the value of data analysis has entered com-
mon culture,with numerous companies showing how sophis-
ticated data analysis leads to cost savings and even direct
revenue.The end result of these opportunities is a grassroots
move to collect and leverage data in multiple organizational
units.While this has many benets in fostering eciency
and data-driven culture [15],it adds to the force of data de-
centralization that data warehousing is supposed to combat.
In this changed climate of widespread,large-scale data
collection,there is a premium on what we dub MAD anal-
ysis skills.The acronym arises from three aspects of this
environment that dier from EDWorthodoxy:
Magnetic:Traditional EDWapproaches\repel"new
data sources,discouraging their incorporation until
they are carefully cleansed and integrated.Given the
ubiquity of data in modern organizations,a data ware-
house can keep pace today only by being\magnetic":
attracting all the data sources that crop up within an
organization regardless of data quality niceties.
Agile:Data Warehousing orthodoxy is based on long-
range,careful design and planning.Given growing
numbers of data sources and increasingly sophisticated
and mission-critical data analyses,a modern warehouse
must instead allow analysts to easily ingest,digest,
produce and adapt data at a rapid pace.This requires
a database whose physical and logical contents can be
in continuous rapid evolution.
Deep:Modern data analyses involve increasingly so-
phisticated statistical methods that go well beyond the
rollups and drilldowns of traditional BI.Moreover,an-
alysts often need to see both the forest and the trees in
running these algorithms { they want to study enor-
mous datasets without resorting to samples and ex-
tracts.The modern data warehouse should serve both
as a deep data repository and as a sophisticated algo-
rithmic runtime engine.
As noted by Varian,there is a growing premium on an-
alysts with MAD skills in data analysis.These are often
highly trained statisticians,who may have strong software
skills but would typically rather focus on deep data analy-
sis than database management.They need to be comple-
mented by MAD approaches to data warehouse design and
database system infrastructure.These goals raise interest-
ing challenges that are dierent than the traditional focus
in the data warehousing research and industry.
1.1 Contributions
In this paper,we describe techniques and experiences we
have developed in our development of MADanalytics for Fox
Audience Network,using a large installation of the Green-
plum Database system.We discuss our database design
methodology that focuses on enabling an agile yet organized
approach to data analysis (Section 4).We present a number
of data-parallel statistical algorithms developed for this set-
ting,which focus on modeling and comparing the densities of
distributions.These include specic methods like Ordinary
Least Squares,Conjugate Gradiant,and Mann-Whitney U
Testing,as well as general purpose techniques like matrix
multiplication and Bootstrapping (Section 5).Finally,we
re ect on critical database system features that enable agile
design and exible algorithm development,including high-
performance data ingress/egress,heterogeneous storage fa-
cilities,and exible programming via both extensible SQL
and MapReduce interfaces to a single system (Section 6).
Underlying our discussion are challenges to a number of
points of conventional wisdom.In designing and analyzing
data warehouses,we advocate the theme\Model Less,It-
erate More".This challenges data warehousing orthodoxy,
and argues for a shift in the locus of power from DBAs to-
ward analysts.We describe the need for unied systems
that embrace and integrate a wide variety of data-intensive
programming styles,since analysts come from many walks
of life.This involves moving beyond religious debates about
the advantages of SQL over MapReduce,or R over Java,to
focus on evolving a single parallel data ow engine that can
support a diversity of programming styles,to tackle sub-
stantive statistical analytics.Finally,we argue that many
data sources and storage formats can and should be knit-
ted together by the parallel data ow engine.This points
toward more uid integration or consolidation of tradition-
ally diverse tools including traditional relational databases,
column stores,ETL tools,and distributed le systems.
Data analytics is not a new area.In this section we de-
scribe standard practice and related work in Business Intel-
ligence and large-scale data analysis,to set the context for
our MAD approach.
2.1 OLAP and Data Cubes
Data Cubes and On-Line Analytic Processing (OLAP)
were popularized in the 1990's,leading to intense commer-
cial development and signicant academic research.The
SQL CUBE BY extension translated the basic idea of OLAP
into a relational setting [8].BI tools package these sum-
maries into fairly intuitive\cross-tabs"visualizations.By
grouping on few dimensions,the analyst sees a coarse\roll-
up"bar-chart;by grouping on more dimensions they\drill
down"into ner grained detail.Statisticians use the phrase
descriptive statistics for this kind of analysis,and tradition-
ally apply the approach to the results of an experimental
study.This functionality is useful for gaining intuition about
a process underlying the experiment.For example,by de-
scribing the clickstream at a website one can get better inu-
ition about underlying properties of the user population.
By contrast,inferential or inductive statistics try to di-
rectly capture the underlying properties of the population.
This includes tting models and parameters to data and
computing likelihood functions.Inferential statistics require
more computation than the simple summaries provided by
OLAP,but provide more probabilistic power that can be
used for tasks like prediction (e.g.,\which users would be
likely to click on this new ad?"),causality analysis (\what
features of a page result in user revisits?"),and distribu-
tional comparison (e.g.,\how do the buying patterns of
truck owners dier from sedan owners?") Inferential ap-
proaches are also more robust to outliers and other par-
ticulars of a given dataset.While OLAP and Data Cubes
remain useful for intuition,the use of inferential statistics
has become an imperative in many important automated
or semi-automated business processes today,including ad
placement,website optimization,and customer relationship
2.2 Databases and Statistical Packages
BI tools provide fairly limited statistical functionality.It
is therefore standard practice in many organizations to ex-
tract portions of a database into desktop software packages:
statistical package like SAS,Matlab or R,spreadsheets like
Excel,or custom code written in languages like Java.
There are various problems with this approach.First,
copying out a large database extract is often much less e-
cient than pushing computation to the data;it is easy to get
orders of magnitude performance gains by running code in
the database.Second,most stat packages require their data
to t in RAM.For large datasets,this means sampling the
database to form an extract,which loses detail.In modern
settings like advertising placement,microtargeting requires
an understanding of even small subpopulations.Samples
and synopses can lose the\long tail"in a data set,and that
is increasingly where the competition for eectiveness lies.
A better approach is to tightly integrate statistical com-
putation with a massively parallel database.Unfortunately,
many current statistical packages do not provide parallel im-
plementations of any kind.Those statistical libraries that
have been parallelized,e.g.,via ScaLAPACK [2],rely on
MPI-based message-passing protocols across processors,and
do not integrate naturally with the data ow parallelism of
popular data-intensive solutions.
2.3 MapReduce and Parallel Programming
While BI and EDW methodologies were being packaged
for enterprise settings,the MapReduce programming model
popularized by Google has captured the attention of many
developers.Google's very visible successes in ad placement
and text processing { and their public embrace of statisti-
cal machine learning { has driven this phenomenon forward
quickly.A recent paper on implementing machine learning
algorithms in MapReduce [3] highlighted a number of stan-
dard techniques that can be computed in a data-parallel
fashion via summations.The Apache Mahout project is
an eort to implement these techniques in the open-source
Hadoop MapReduce engine.The observation of that pa-
per applies equally well to SQL,but the technical-social
phenomenon surrounding MapReduce is important:it has
caused a number of statistically-minded researchers and de-
velopers to focus on Big Data and data-parallel program-
ming,rather than on multiprocessing via MPI.This spirit
of data-parallel programming informs the design of our algo-
rithms in Section 5 as well.But programming style is only
one aspect of a MAD approach to managing the process of
analytics,as we describe in Section 4.
2.4 DataMiningandAnalytics inthe Database
There is a signicant literature on parallel data mining al-
gorithms;see for example the collection by Zaki and Ho [24].
The most common data mining techniques { clustering,clas-
sication,association rules { concern themselves with what
we might call pointwise decisions.Classiers and clustering
assign individual points to cohorts (class labels or cluster
IDs);association rules form combinatorial collections of in-
dividual points at the output.Though these problems are
non-trivial,the eld of statistical modeling covers quite a bit
more ground in addition.For example,a common technique
in advertising analysis is A=B testing,which takes response
rates of a subpopulation and a control group,and compares
their statistical densities on various metrics.
Standard data mining methods in commercial databases
are useful but quite targeted:they correspond to only a
small number of the hundreds of statistical libraries that
would ship with a stat package like R,SAS,or Matlab.
Moreover,they are typically\black box"implementations,
whose code is compiled into an engine plugin.By contrast,
statistical package like Ror Matlab are exible programming
environments,with library routines that can be modied
and extended by analysts.MAD analysis requires similar
programming abilities to be brought to Big Data scenar-
ios,via extensible SQL and/or MapReduce.Black-box data
mining routines can sometimes be useful in that context,
but only in a modest fraction of cases.
In addition to our work,there are other interesting eorts
to do signicant scientic computation in SQL documented
in the literature,most notably in the Sloan Digital Sky Sur-
vey [22].The management of experiments [14] and complex
SQL code [16] are also relevant.The interested reader is also
referred to new research systems on scientic data manage-
ment [21] and scalability for R[25].The idea of\dataspaces"
is related to our MAD philosophy as well,with a focus on
data integration [6].
Fox Audience Network serves ads across several Fox online
publishers,including MySpace.com,IGN.comand Scout.com.
With a reach of over one hundred and fty million users,it
is one of the world's largest ad networks.
The FAN Data Warehouse today is implemented via the
Greenplum Database system running on 42 nodes:40 for
query processing,and two master nodes (one for failover).
The query processing nodes are Sun X4500 (\Thumper")
machines,congured with 2 dual-core Opteron processors,
48 500-GB drives,and 16 GB of RAM.The FAN EDW
currently holds over 200 terabytes of unique production data
that is then mirrored for failover.It is growing rapidly:
every day the FAN data warehouse ingests four to seven
billion rows of ad server logs amounting to approximately
ve terabytes.The major impression fact table stretches
back to October of 2007,creating a single table of over 1.5
trillion rows.FAN's Customer Relationship Management
(CRM) solution also provides millions of rows of advertiser
and campaign dimension data every day.Additionally there
is extensive data on each of over 150 million users.The FAN
EDWis the sole repository where all three data sources are
integrated for use by research or reporting teams.
The EDW is expected to support very disparate users,
from sales account managers to research scientists.These
users'needs are very dierent,and a variety of reporting and
statistical software tools are leveraged against the warehouse
every day.The MicroStrategy BI tool has been implemented
directly against the warehouse to service Sales,Marketing
and most other basic reporting needs.Research scientists
also have direct command-line access to the same warehouse.
Thus,the query ecosystem is very dynamic.
No set of pre-dened aggregates could possibly cover every
question.For example,it is easy to imagine questions that
combine both advertiser and user variables.At FAN,this
typically means hitting the fact tables directly.For instance,
one question that is easy for a salesperson to pose is:How
many female WWF enthusiasts under the age of 30 visited
the Toyota community over the last four days and saw a
medium rectangle?(A\medium rectangle"is a standard-
sized web ad.) The goal is to provide answers within minutes
to such ad hoc questions,and it is not acceptable to refuse
to answer questions that were not precomputed.
This example is satised by a simple SQL query,but the
follow-up question in such scenarios is invariably a compar-
ative one:How are these people similar to those that visited
Nissan?At this stage we begin to engage in open-ended
multi-dimensional statistical analysis requiring more sophis-
ticated methods.At FAN,R is a popular choice for research
and the RODBC package is often used to interface directly
with the warehouse.When these questions can be reduced
to fewer than 5 million records,the data is often exported
and analyzed in R.The cases where it cannot be reduced
form a main impetus for this research paper.
In addition to the data warehouse,the machine learning
teamat FANalso makes use of several large Hadoop clusters.
That team employs dozens of algorithms for classication,
supervised and unsupervised learning,neural network and
natural language processing.These are not techniques tra-
ditionally addressed by an RDBMS,and the implementation
in Hadoop results in large data migration eorts for specic
single-purpose studies.The availability of machine learning
methods directly within the warehouse would oer a signi-
cant savings in time,training,and systemmanagement,and
is one of the goals of the work described here.
Traditional Data Warehouse philosophy revolves around a
disciplined approach to modeling information and processes
in an enterprise.In the words of warehousing advocate Bill
Inmon,it is an\architected environment"[13].This view
of warehousing is at odds with the magnetism and agility
desired in many new analysis settings,as we describe below.
4.1 New Requirements
As data savvy people,analysts introduce a new set of re-
quirements to a database environment.They have a deep
understanding of the enterprise data and tend to be early
adopters of new data sources.In the same way that sys-
tems engineers always want the latest-and-greatest hardware
technologies,analysts are always hungry for new sources
of data.When new data-generating business processes are
launched,analysts demand the new data immediately.
These desires for speed and breadth of data raise ten-
sions with Data Warehousing orthodoxy.Inmon describes
the traditional view:
There is no point in bringing data...into the
data warehouse environment without integrating
it.If the data arrives at the data warehouse in an
unintegrated state,it cannot be used to support
a corporate view of data.And a corporate view
of data is one of the essences of the architected
Unfortunately,the challenge of perfectly integrating a new
data source into an\architected"warehouse is often sub-
stantial,and can hold up access to data for months { or in
many cases,forever.The architectural view introduces fric-
tion into analytics,repels data sources from the warehouse,
and as a result produces shallow incomplete warehouses.It
is the opposite of the MAD ideal.
Given the growing sophistication of analysts and the grow-
ing value of analytics,we take the view that it is much more
important to provide agility to analysts than to aspire to
an elusive ideal of full integration.In fact,analysts serve as
key data magnets in an organization,scouting for interest-
ing data that should be part of the corporate big picture.
They can also act as an early warning system for data qual-
ity issues.For the privilege of being the rst to see the
data,they are more tolerant of dirty data,and will act to
apply pressure on operational data producers upstream of
the warehouse to rectify the data before it arrives.Analysts
typically have much higher standards for the data than a
typical business unit working with BI tools.They are un-
daunted by big, at fact tables that hold complete data sets,
scorning samples and aggregates,which can both mask er-
rors and lose important features in the tails of distributions.
Hence it is our experience that a good relationship with
the analytics team is an excellent preventative measure for
data management issues later on.Feeding their appetites
and responding to their concerns improves the overall health
of the warehouse.
Ultimately,the analysts produce new data products that
are valuable to the enterprise.They are not just consumers,
but producers of enterprise data.This requires the ware-
house to be prepared to\productionalize"the data gener-
ated by the analysts into standard business reporting tools.
It is also useful,when possible,to leverage a single par-
allel computing platform,and push as much functionality
as possible into it.This lowers the cost of operations,and
eases the evolution of software from analyst experiments to
production code that aects operational behavior.For ex-
ample,the lifecycle of an ad placement algorithmmight start
in a speculative analytic task,and end as a customer fac-
ing feature.If it is a data-driven feature,it is best to have
that entire lifecycle focused in a single development environ-
ment on the full enterprise dataset.In this respect we agree
with a central tenet of Data Warehouse orthodoxy:there is
tangible benet to getting an organization's data into one
repository.We dier on how to achieve that goal in a useful
and sophisticated manner.
In sum,a healthy business should not assume an archi-
tected data warehouse,but rather an evolving structure that
iterates through a continuing cycle of change:
1.The business performs analytics to identify areas of
potential improvement.
2.The business either reacts to or ignores this analysis.
3.Areaction results in new or dierent business practices
{ perhaps new processes or operational systems { that
typically generate new data sets.
4.Analysts incorporate new data sets into their models.
5.The business again asks itself\How can we improve?"
A healthy,competitive business will look to increase the
pace of this cycle.The MAD approach we describe next is
a design pattern for keeping up with that increasing pace.
4.2 Getting More MAD
The central philosophy in MAD data modeling is to get
the organization's data into the warehouse as soon as possi-
ble.Secondary to that goal,the cleaning and integration of
the data should be staged intelligently.
To turn these themes into practice,we advocate the three-
layer approach.A Staging schema should be used when
loading raw fact tables or logs.Only engineers and some
analysts are permitted to manipulate data in this schema.
The Production Data Warehouse schema holds the aggre-
gates that serve most users.More sophisticated users com-
fortable in a large-scale SQL environment are given access
to this schema.A separate Reporting schema is maintained
to hold specialized,static aggregates that support reporting
tools and casual users.It should be tuned to provide rapid
access to modest amounts of data.
These three layers are not physically separated.Users
with the correct permissions are able to cross-join between
layers and schemas.In the FAN model,the Staging schema
holds raw action logs.Analysts are given access to these
logs for research purposes and to encourage a laboratory
approach to data analysis.Questions that start at the event
log level often become broader in scope,allowing custom
aggregates.Communication between the researchers and
the DBAs uncovers common questions and often results in
aggregates that were originally personalized for an analyst
being promoted into the production schema.
The Production schema provides quick answers to com-
mon questions that are not yet so common as to need re-
ports.Many of these are anticipated during installation,
but many are not.Data tends to create\feeding frenzies"
as organizations,starved for data only months before,be-
gin launching question after question at the database.Being
nimble in this stage is crucial,as the business analysts begin
to enter the environment.They want to know things at the
daily or monthly level.Questions about daily performance
are turned into dashboards.
Analysts should also be given a fourth class of schema
within the warehouse,which we call a\sandbox"
sandbox schema is under the analysts'full control,and is
to be used for managing their experimental processes.Ana-
lysts are data-savvy developers,and they often want to track
and record their work and work products in a database.For
example,when developing complex SQL like we will see in
Section 5,it is typical to use SQL views as\subroutines"to
structure a coding task.During development,these views
are likely to be dened and edited frequently.Similarly,ana-
lysts may want to materialize query results as they are doing
their work,and reuse the results later;this materialization
can also help the software engineering process,and improve
eciency during iterative design of an analysis work ow.In
addition,analysts often want to squirrel away pet data sets
for their own convenience over time,to use in prototyping
new techniques on known inputs.
The ability to leap from very specic to very general en-
courages investigation and creativity.As the data gets used,
transformed,discussed and adopted,the organization learns
and changes its practices.The pace of progress is bound by
the pace and depth of investigation.MADdesign is intended
to accelerate this pace.
Analysts and statisticians are an organization's most data-
savvy agents,and hence key to its MAD skills.In this sec-
tion,we focus on powerful,general statistical methods that
make the data warehouse more\Magnetic"and\Agile"for
these analysts,encouraging themto go\Deep"and substan-
tially increase the sophistication and scale of data analytics.
Our general approach is to develop a hierarchy of math-
ematical concepts in SQL,and encapsulate them in a way
that allows analysts to work in relatively familiar statistical
terminology,without having to develop statistical methods
in SQL from rst principles for each computation.Similar
functionality could be coded up in a MapReduce syntax.
Traditional SQL databases provide data types and func-
tions for simple (scalar) arithmetic.The next layer of ab-
straction is vector arithmetic,which brings its own suite of
operators.Vector objects combined with vector operators
bring us the language of linear algebra.We suggest methods
for these operators in Section 5.1.It is this level that allows
us to speak in the language of machine learning,mathemat-
ical modeling,and statistics.The next layer of abstraction
is the function level;probability densities are specialized
functions.Inductively,there is another layer of abstraction,
where functions are the base objects and algebras are cre-
This is not to be confused with\sandboxing"software pro-
cesses for computer security.Our usage is intended to con-
vey a sense of play.
ated using operators called\functionals"acting upon func-
tions.This is the realm of functional analysis.Methods like
ttests or likelihood ratios are functionals.A=B testing in-
volves functionals,treating two mathematical objects at a
time:probability density functions f
() and f
Our job,then,is to advance database methods fromscalars
to vectors to functions to functionals.Further,we must do
this in a massively parallel environment.This is not triv-
ial.Even the apparently\simple"problem of representing
matrices does not have a unique optimal solution.In the
next few sections,we describe methods we have used to con-
vince a parallel database to behave like a massively scalable
statistical package.We begin with vector arithmetic and
work toward functionals,with additional powerful statisti-
cal methods along the way.
5.1 Vectors and Matrices
Relational databases are designed to scale with cardinal-
ity.We describe how we have represented large\vector"
and\matrix"objects as relations,and implemented basic
operations for them.This gives us vector arithmetic.
Before constructing operators,we need to dene what a
vector (often in the form of a matrix) would mean in the
context of a database.There are many ways to partition
(\block",\chunk") such matrices across nodes in a par-
allel system (e.g.,see [2],Chapter 4).A simple scheme
that ts well with parallel databases is to represent the ma-
trix as a relation with schema (row
number integer,vector
numeric[]) and allow the DBMS to partition the rows across
processors arbitrarily { e.g.via hashing or a round-robin
scheme.In practice,we often materialize both A and A
to allow this row-representation method to work more e-
Given matrices represented as horizontally partitioned re-
lations in this manner,we need to implement basic matrix
arithmetic as queries over those relations,which can be ex-
ecuted in a parallel fashion.
Consider two matrices A and B of identical dimensions.
Matrix addition A+B is easy to express in SQL:
SELECT A.row_number,A.vector + B.vector
WHERE A.row_number = B.row_number;
Note that the + operator here is operating on arrays of nu-
meric types and returns a similarly-dimensioned array,so
the output is a matrix whose dimensions are equal to the
inputs.If vector addition is not provided by the DBMS,it
is very easy to implement via object-relational extensions
and register as an inx operator [19].A query optimizer is
likely to choose a hash join for this query,which parallelizes
Multiplication of a matrix and a vector Av is also simple:
SELECT 1,array_accum(row_number,vector*v) FROM A;
Again,the * operator here is operating on arrays of numer-
ics,but in this case returns a single numeric value { the
dot product of its inputs ~x ~y =
.This too can be
implemented via a user-dened function,and registered as
an inx operator with the query language [19].The pairs
number,vector*v) represent a vector as (index,value)
pairs.To get back to our canonical row-wise representation,
we convert to an array type in a single output row via the
custom aggregation function array
accum(x,v),which accu-
mulates a single array eld,setting position x to value v for
each row of input.
Most RDBMSs have sequence functions.In PostgreSQL
and Greenplum,the command generate
series(1,50) will
generate 1;2;:::50.One way to compute a matrix transpose
of an mn matrix A is expressible as follows (for n = 3):
SELECT S.col_number,
FROM A,generate_series(1,3) AS S(col_number)
GROUP BY S.col_number;
Unfortunately if A stores n-dimensional vectors,then this
results in up to n copies of the table A being fed into the
grouping operator.An alternative is to convert to a dier-
ent matrix representation,for example a sparse represen-
tation of the form (row
advantage to this approach is that the SQL is much easier
to construct for multiplication of matrices AB.
SELECT A.row_number,B.column_number,SUM(A.value * B.value)
WHERE A.column_number = B.row_number
GROUP BY A.row_number,B.column_number
This query is very ecient on sparse matrices,as 0 values
would not be stored.In general,it is well-known in par-
allel computation that no single storage representation will
service all needs,and in practice blends need to be used.
Without proper abstractions,this can lead to confusion as
custom operators need to be dened for each representa-
tion.In SQL,dierent representations can be achieved via
naming conventions on (materialized) views,but the choice
of views typically is done by the analyst,since a traditional
query optimizer is unaware of the equivalence of these views.
Work on this front can be found in the parallel computing
literature [23],but has yet to be integrated with relational
query optimization and data-intensive computing.
The above conversation applies to scalar multiplication,
vector addition and vector/matrix multiplication,which are
essentially single-pass methods.The task of matrix division
is not denitively solved in a parallel context.One awkard-
ness in SQL is the lack of convenient syntax for iteration.
The fundamental routines for nding a matrix inverse in-
volve two or more passes over the data.However,recursive
or iterative procedures can be driven by external processes
with a minimal amount of data ow over the master node.
For instance,in the conjugate gradient method described in
Section 5.2.2,only a single value is queried between itera-
tions.Although matrix division is complicated,we are able
to develop the rest of our methods in this paper via pseudo-
inverse routines (with textbook math programming caveats
on existence and convergence.)
A comprehensive suite of (now distributed) vector objects
and their operators generate the nouns and verbs of statisti-
cal mathematics.From there,functions follow as sentences.
We continue with a familiar example,cast in this language.
5.1.1 tf-idf and Cosine Similarity
The introduction of vector operations allows us to speak
much more compactly about methods.We consider a spe-
cic example here:document similarity,a common tool in
web advertising.One usage is in fraud detection.When
many dierent advertisers link to pages that are very sim-
ilar,it is typical that they are actually the same malicious
party,and very likely using stolen credit cards for payment.
It is therefore wise to follow advertisers'outlinks and look
for patterns of similar documents.
The common document similarity metric tf-idf involves
three or four steps,all of which can be easily distributed and
lend themselves very well to SQL methods.First,triples of
(document,term,count) must be created.Then marginals
along document and term are calculated,via simple SQL
GROUP BY queries.Next each original triple can be expanded
with a tf-idf score along every\dimension"{ i.e.,for ev-
ery word in the resulting dictionary { by joining the triples
with the document marginals and dividing out the document
counts from the term counts.From there,the cosine simi-
larity of two document vectors is calculated on tf-idf scores
and the standard\distance"metric is obtained.
Specically,it is well known that given two term-weight
vectors x and y,the cosine similarity is given by =
.It is not dicult to construct\vanilla"SQL to
reproduce this equation.But analysts with backgrounds in
statistics do not think that way { this approach (and the
sparse matrix example of Section 5.1) focuses on pairing up
scalar values,rather than higher-level\whole-object"rea-
soning on vectors.Being able to express ideas like tf-idf in
terms of linear algebra lowers the barrier to entry for statis-
ticians to program against the database.The dot-product
operator reduces tf-idf to a very natural syntax.Suppose A
has one row per document vector.
SELECT a1.row_id AS document_i,a2.row_id AS document_j,
(a1.row_v * a2.row_v)/
((a1.row_v * a1.row_v) * (a2.row_v * a2.row_v)) AS theta
FROM a AS a1,a AS a2
WHERE a1.row_id > a2.row_id
To any analyst comfortable with scripting environments
such as SAS or R,this formation is perfectly acceptable.Fur-
ther,the DBA has done the work of distributing the objects
and dening the operators.When the objects and operators
become more complicated,the advantages to having pre-
dened operators increases.From a practical standpoint,
we have moved the database closer to an interactive,an-
alytic programming environment and away from a simple
data retrieval system.
5.2 Matrix Based Analytical Methods
The matrices of primary interest in our setting are large,
dense matrices.A common subject is a distance matrix D
where D(i;j) > 0 for almost all (i;j).Another theme is
covariance matrices in tightly correlated data sets.
5.2.1 Ordinary Least Squares
We begin with Ordinary Least Squares (OLS),a classical
method for tting a curve to data,typically with a poly-
nomial function.In web advertising,one standard appli-
cation is in modeling seasonal trends.More generally,in
many ad-hoc explorations it is where analysis starts.Given
a few simple vector-oriented user-dened functions,it be-
comes natural for an analyst to express OLS in SQL.
In our case here we nd a statistical estimate of the pa-
best satisfying Y = X.Here,X = n k is
a set of xed (independent) variables,and Y is a set of n
observations (dependent variables) that we wish to model
via a function of X with parameters .
As noted in [3],
y (1)
can be calculated by computing A = X
X and b = X
y as
summations.In a parallel database this can be executed by
having each partition of the database calculate the local A
and b in parallel and then merge these intermediate results
in a nal sequential calculation.
This will produce a square matrix and a vector based on
the size of the independent variable vector.The nal cal-
culation is computed by inverting the small A matrix and
multiplying by the vector to derive the coecients
Additionally,calculation of the coecient of determina-
tion R
can be calculated concurrently by
SSR = b
In the following SQL query,we compute the coecients
,as well as the components of the coecient of determi-
CREATE VIEW ols AS
SELECT pseudo_inverse(A) * b as beta_star,
(transpose(b) * (pseudo_inverse(A) * b)
- sum_y2/count) -- SSR
/(sum_yy - sumy2/n) -- TSS
as r_squared
SELECT sum(transpose(d.vector) * d.vector) as A,
sum(d.vector * y) as b,
sum(y)^2 as sum_y2,sum(y^2) as sum_yy,
count(*) as n
FROM design d
) ols_aggs;
Note the use of a user-dened function for vector transpose,
and user-dened aggregates for summation of (multidimen-
sional) array objects.The array A is a small in-memory
matrix that we treat as a single object;the pseudo-inverse
function implements the textbook Moore-Penrose pseudoin-
verse of the matrix.
All of the above can be eciently calculated in a single
pass of the data.For convenience,we encapsulated this yet
further via two user-dened aggregate functions:
SELECT ols_coef(d.y,d.vector),ols_r2(d.y,d.vector)
FROM design d;
Prior to the implementation of this functionality within
the DBMS,one Greenplumcustomer was accustomed to cal-
culating the OLS by exporting data and importing the data
into R for calculation,a process that took several hours to
complete.They reported signicant performance improve-
ment when they moved to running the regression within the
DBMS.Most of the benet derived from running the analy-
sis in parallel close to the data with minimal data movement.
5.2.2 Conjugate Gradient
In this subsection we develop a data-parallel implementa-
tion of the Conjugate Gradiant method for solving a system
of linear equations.We can use this to implement Sup-
port Vector Machines,a state-of-the-art technique for binary
classication.Binary classiers are a common tool in mod-
ern ad placement,used to turn complex multi-dimensional
user features into simple boolean labels like\is a car en-
thusiast"that can be combined into enthusiast charts.In
addition to serving as a building block for SVMs,the Conju-
gate Gradiant method allows us to optimize a large class of
functions that can be approximated by second order Taylor
To a mathematician,the solution to the matrix equation
Ax = b is simple when it exists:x = A
b.As noted in
Section 5.1,we cannot assume we can nd A
.If matrix
A is n n symmetric and positive denite (SPD),we can
use the Conjugate Gradient method.This method requires
neither df(y) nor A
and converges in no more than n
interations.A general treatment is given in [17].Here we
outline the solution to Ax = b as an extremum of f(x) =
Ax + b
x + c.Broadly,we have an estimate ^x to our
solution x
.Since ^x is only an estimate,r
= A^x b is
non-zero.Subtracting this error r
from the estimate allows
us to generate a series p
= r
fA^x bg of orthogonal
vectors.The solution will be x
below.We end at the point kr
< for a suitable .
There are several update algorithms,we have written ours
in matrix notation.
= b A^x
= r
;i = 0
Begin iteration over i.
= x
= r
check kr
= r
To incorporate this method into the database,we stored
) as a rowand inserted rowi+1 in one pass.This
required the construction of functions update
A),and update
i,A).Though the function
calls were redundant (for instance,update
v() also runs the
update of r
),this allowed us to insert one full row at a
time.An external driver process then checks the value of r
before proceeding.Upon convergence,it is rudimentary to
compute x
The presence of the conjugate gradient method enables
even more sophisticated techniques like Support Vector Ma-
chines (SVM).At their core,SVMs seek to maximize the
distance between a set of points and a candiate hyperplane.
This distance is denoted by the magnitude of the normal
vectors kwk
.Most methods incorporate the integers f0;1g
as labels c,so the problem becomes
f(w) =
;subject to c
w b 0:
This method applies to the more general issue of high
dimensional functions under a Taylor expansion f
) + df(x)(x x
) +
(x x
f(x)(x x
) With a
good initial guess for x
and the common assumption of
continuity of f(),we know the the matrix will be SPD near
.See [17] for details.
5.3 Functionals
Basic statistics are not new to relational databases { most
support means,variances and some form of quantiles.But
modeling and comparative statistics are not typically built-
in functionality.In this section we provide data-parallel
implementations of a number of comparative statistics ex-
pressed in SQL.
In the previous section,scalars or vectors were the atomic
unit.Here a probability density function is the founda-
tional object.For instance the Normal (Gaussian) density
f(x) = e
is considered by mathematicians as a
single\entity"with two attributes:the mean and vari-
ance .A common statistical question is to see how well a
data set ts to a target density function.The zscore of
a datum x is given z(x) =
and is easy to obtain in
standard SQL.
SELECT x.value,(x.value - d.mu) * d.n/d.sigma AS z_score
FROM x,design d
5.3.1 Mann-Whitney U Test
Rank and order statistics are quite amenable to relational
treatments,since their main purpose is to evaluate a set of
data,rather then one datum at a time.The next example
illustrates the notion of comparing two entire sets of data
without the overhead of describing a parameterized density.
The Mann-Whitney U Test (MWU) is a popular substi-
tute for Student's t-test in the case of non-parametric data.
The general idea it to take two populations A and B and
decide if they are from the same underlying population by
examining the rank order in which members of A and B
show up in a general ordering.The cartoon is that if mem-
bers of A are at the\front"of the line and members of
B are at the\back"of the line,then A and B are dier-
ent populations.In an advertising setting,click-through
rates for web ads tend to defy simple parametric models
like Gaussians or log-normal distributions.But it is often
useful to compare click-through rate distributions for dier-
ent ad campaigns,e.g.,to choose one with a better median
click-through.MWU addresses this task.
Given a table T with columns SAMPLE
ID,VALUE,row num-
bers are obtained and summed via SQL windowing func-
SELECT sample_id,avg(value) AS sample_avg
sum(rown) AS rank_sum,count(*) AS sample_n,
sum(rown) - count(*) * (count(*) + 1) AS sample_us
FROM (SELECT sample_id,row_number() OVER
(ORDER BY value DESC) AS rown,
FROM T) AS ordered
GROUP BY sample_id
Assuming the condition of large sample sizes,for instance
greater then 5,000,the normal approximation can be justi-
ed.Using the previous view R,the nal reported statistics
are given by
SELECT r.sample_u,r.sample_avg,r.sample_n
(r.sample_u - a.sum_u/2)/
sqrt(a.sum_u * (a.sum_n + 1)/12) AS z_score
FROM R as r,(SELECT sum(sample_u) AS sum_u,
sum(sample_n) AS sum_n
FROM R) AS a
GROUP BY r.sample_u,r.sample_avg,r.sample_n,
The end result is a small set of numbers that describe a
relationship of functions.This simple routine can be en-
capsulated by stored procedures and made available to the
analysts via a simple SELECT mann
whitney(value) FROM table
call,elevating the vocabulary of the database tremendously.
5.3.2 Log-Likelihood Ratios
Likelihood ratios are useful for comparing a subpopula-
tion to an overall population on a particular attributed.As
an example in advertising,consider two attributes of users:
beverage choice,and family status.One might want to know
whether coee attracts new parents more than the general
This is a case of having two density (or mass) functions
for the same data set X.Denote one distribution as null
hypothesis f
and the other as alternate f
and f
are dierent parameterizations of the same density.
For instance,N(
) and N(
).The likelihood L
under f
is given by
= L(Xjf
) =
The log-likelihood ratio is given by the quotient 2 log (L
Taking the log allows us to use the well-known
mation for large n.Also,the products turn nicely into sums
and an RDBMS can handle it easily in parallel.
log f
) 2
log f
This calculation distributes nicely if f
:R!R,which most
do.If f
!R,then care must be taken in managing
the vectors as distributed objects.Suppose the values are
in table T and the function f
() has been written as a user-
dened function f
llk(x numeric,param numeric).Then the
entire experiment is can be performed with the call
SELECT 2 * sum(log(f_llk(T.value,d.alt_param))) -
2 * sum(log(f_llk(T.value,d.null_param))) AS llr
FROM T,design AS d
This represents a signicant gain in exibility and sophisti-
cation for any RDBMS.
Example:The Multinomial Distribution
The multinomial distribution extends the binomial dis-
tribution.Consider a random variable X with k discrete
outcomes.These have probabilities p = (p
).In n
trials,the joint probability distribution is given by
P(Xjp) =
To obtain p
,we assume a table outcome with column
outcome representing the base population.
SELECT outcome,
outcome_count/sum(outcome_count) over () AS p
FROM (SELECT outcome,count(*)::numeric AS outcome_count
FROM input
GROUP BY outcome) AS a
In the context of model selection,it is often convenient to
compare the same data set under two dierent multinomial
LLR = 2 log
= 2 log
= 2
log ~p
log p
Or in SQL:
SELECT 2 * sum(T.outcome_count * log B.p)
- 2 * sum(T.outcome_count * log T.p)
FROM B,test_population AS T
WHERE B.outcome = T.outcome
5.4 Resampling Techniques
Parametric modeling assumes that data comes from some
process that is well-represented by mathematical models
with a number of parameters { e.g.,the mean and variance
of a Normal distribution.The parameters of the real-world
process need to be estimated from existing data that serves
as a\sample"of that process.One might be tempted to
simply use the SQL AVERAGE and STDDEV aggregates to do
this on a big dataset,but that is typically not a good idea.
Large real-world datasets like those at FAN invariable con-
tain outlier values and other artifacts.Naive\sample statis-
tics"over the data { i.e.,simple SQL aggregates { are not
robust [10],and will\overt"to those artifacts.This can
keep the models from properly representing the real-world
process of interest.
The basic idea in resampling is to repeatedly take samples
of a data set in a controlled fashion,compute a summary
statistic over each sample,and carefully combine the sam-
ples to estimate of a property of the entire data set more
robustly.Intuitively,rare outliers will appear in few or no
samples,and hence will not perturb the estimators used.
There are two standard resampling techniques in the statis-
tics literature.The bootstrap method is straightforward:
from a population of size N,pick k members (a subsam-
ple of size k) from the population and compute the desired
.Now replace your subsample and pick another
random k members.Your new statistic
will be dierent
from your previous statistic.Repeat this\sampling"pro-
cess tens of thousands of times.The distribution of the re-
's is called the sampling distribution.The Central
Limit Theorem says that the sampling distribution is nor-
mal,so the mean of a large sampling distribution produces
an accurate measure
.The alternative to bootstrapping is
the jackknife method,which repeatedly recomputes a sum-
mary statistic
by leaving out one or more data items from
the full data set to measure the in uence of certain sub-
populations.The resulting set of observations is used as a
sampling distribution in the same way as in bootstrapping,
to generate a good estimator for the underlying statistic of
Importantly,it is not required that all subsamples be of
the exact same size,though widely disparate subsample sizes
can lead to incorrect error bounds.
Assuming the statistic of interest is easy to obtain via
SQL,for instance the average of a set of values,then the only
work needing to be done is to orchestrate the resampling
via a suciently random number generator.We illustrate
this with an example of bootstrapping.Consider a table
T with two columns (row
id,value) and N rows.Assume
that the row
id column ranges densely from1:::N.Because
each sample is done with replacement,we can pre-assign the
subsampling.That is,if we do M samples,we can decide in
advance that record i will appear in subsamples 1;2;25;:::
The function random() generates a uniformly random ele-
ment of (0;1) and floor(x) truncates to the integer portion
of x.We use these functions to design a resampling exper-
iment.Suppose we have N = 100 subjects and we wish to
have 10;000 trials each with subsample size 3.
CREATE VIEW design AS
SELECT a.trial_id,
floor (100 * random()) AS row_id
FROM generate_series(1,10000) AS a (trial_id),
generate_series(1,3) AS b (subsample_id)
The reliance upon the random number generator here is
system dependent and the researcher needs to verify that
scaling the random() function still returns a uniform random
variable along the scale.Performing the experiment over
the view now takes a single query:
CREATE VIEW trials AS
SELECT d.trial_id,AVG(a.values) AS avg_value
FROM design d,T
WHERE d.row_id = T.row_id
GROUP BY d.trial_id
This returns the sampling distribution:the average values
of each subsample.The nal result of the bootstrapping
process is a simple query over this view:
SELECT AVG(avg_value),STDDEV(avg_value)
FROM trials;
This query returns the statistic of interest after the given
number of resamples.The AVG() and STDDEV() functions
are already done in parallel,so the entire technique is done
in parallel.Note that the design view is relatively small
( 30;000 rows) so it will t in memory;hence all 10;000
\experiments"are performed in a single parallel pass of the
table T;i.e.,roughly the same cost as computing a naive
SQL aggregate.
Jackkning makes use of a similar trick to generate mul-
tiple experiments in a single pass of the table,excluding a
random subpopulation in each.
The MAD approach we sketch in Section 4 requires sup-
port from the DBMS.First,getting data into a\Magnetic"
database must be painless and ecient,so analysts will play
with new data sources within the warehouse.Second,to en-
courage\Agility",the system has to make physical storage
evolution easy and ecient.Finally,\Depth"of analysis
{ and really all aspects of MAD analytics { require the
database to be a powerful, exible programming environ-
ment that welcomes developers of various stripes.
6.1 Loading and Unloading
The importance of high-speed data loading and dumping
for big parallel DBMSs was highlighted over a decade ago [1],
and it is even more important today.Analysts load newdata
sets frequently,and quite often like to\slosh"large data sets
between systems (e.g.,between the DBMS and a Hadoop
cluster) for specic tasks.As they clean and rene data and
engage in developing analysis processes,they iterate over
tasks frequently.If load times are measured in days,the
ow of an analyst's work qualitatively changes.These kinds
of delays repel data from the warehouse.
In addition to loading the database quickly,a good DBMS
for MADanalytics should enable database users to run queries
directly against external tables:raw feeds from les or ser-
vices that are accessed on demand during query processing.
By accessing external data directly and in parallel,a good
DBMS can eliminate the overhead of landing data and keep-
ing it refreshed.External tables (\wrappers") are typically
discussed in the context of data integration [18].But the
focus in a MAD warehouse context is on massively parallel
access to le data that lives on a local high-speed network.
Greenplum implements fully parallel access for both load-
ing and query processing over external tables via a technique
called Scatter/Gather Streaming.The idea is similar to tra-
ditional shared-nothing database internals [7],but requires
coordination with external processes to\feed"all the DBMS
nodes in parallel.As the data is streamed into the system
it can be landed in database tables for subsequent access,
or used directly as a purely external table with parallel I/O.
Using this technology,Greenplum customers have reporting
loading speeds of a fully-mirrored,production database in
excess of four terabytes per hour with negligible impact on
concurrent database operations.
6.1.1 ETL and ELT
Traditional data warehousing is supported by customtools
for the Extract-Transform-Load (ETL) task.In recent years,
there is increasing pressure to push the work of transforma-
tion into the DBMS,to enable parallel execution via SQL
transformation scripts.This approach has been dubbed
ELT since transformation is done after loading.The ELT
approach becomes even more natural with external tables.
Transformation queries can be written against external ta-
bles,removing the need to ever load untransformed data.
This can speed up the design loop for transformations sub-
stantially { especially when combined with SQL's LIMIT clause
as a\poor man's Online Aggregation"[11] to debug trans-
In addition to transformations written in SQL,Green-
plum supports MapReduce scripting in the DBMS,which
can run over either external data via Scatter/Gather,or in-
database tables (Section 6.3).This allows programmers to
write transformation scripts in the data ow-style program-
ming used by many ETL tools,while running at scale using
the DBMS'facilities for parallelism.
6.2 Data Evolution:Storage and Partitioning
The data lifecycle in a MAD warehouse includes data in
various states.When a data source is rst brought into
the system,analysts will typically iterate over it frequently
with signicant analysis and transformation.As transfor-
mations and table denitions begin to settle for a particular
data source,the workload looks more like traditional EDW
settings:frequent appends to large\fact"tables,and occa-
sional updates to\detail"tables.This mature data is likely
to be used for ad-hoc analysis as well as for standard re-
porting tasks.As data in the\fact"tables ages over time,
it may be accessed less frequently or even\rolled o"to an
external archive.Note that all these stages co-occur in a
single warehouse at a given time.
Hence a good DBMS for MAD analytics needs to support
multiple storage mechanisms,targeted at dierent stages of
the data lifecycle.In the early stage,external tables pro-
vide a lightweight approach to experiment with transforma-
tions.Detail tables are often modest in size and undergo
periodic updates;they are well served by traditional trans-
actional storage techniques.Append-mostly fact tables can
be better served by compressed storage,which can handle
appends and reads eciently,at the expense of making up-
dates slower.It should be possible to roll this data o of the
warehouse as it ages,without disrupting ongoing processing.
Greenplum provides multiple storage engines,with a rich
SQL partitioning specication to apply them exibly across
and within tables.As mentioned above,Greenplumincludes
external table support.Greenplum also provides a tradi-
tional\heap"storage format for data that sees frequent
updates,and a highly-compressed\append-only"(AO) ta-
ble feature for data that is not going to be updated;both
are integrated within a transactional framework.Green-
plum AO storage units can have a variety of compression
modes.At one extreme,with compression o,bulk loads run
very quickly.Alternatively,the most aggressive compression
modes are tuned to use as little space as possible.There is
also a middle ground with\medium"compression to provide
improved table scan time at the expense of slightly slower
loads.In a recent version Greenplum also adds\column-
store"partitioning of append-only tables,akin to ideas in
the literature [20].This can improve compression,and en-
sures that queries over large archival tables only do I/O for
the columns they need to see.
A DBA should be able to specify the storage mechanism
to be used in a exible way.Greenplum supports many
ways to partition tables in order to increase query and data
load performance,as well as to aid in managing large data
sets.The top-most layer of partitioning is a distribution
policy specied via a DISTRIBUTED BY clause in the CREATE
TABLE statement that determines how the rows of a table
are distributed across the individual nodes that comprise
a Greenplum cluster.While all tables have a distribution
policy,users can optionally specify a partitioning policy for
a table,which separates the data in the table into parti-
tions by range or list.A range partitioning policy lets users
specify an ordered,non-overlapping set of partitions for a
partitioning column,where each partition has a START and
END value.A list partitioning policy lets users specify a set
of partitions for a collection of columns,where each parti-
tion corresponds to a particular value.For example,a sales
table may be hash-distributed over the nodes by sales
On each node,the rows are further partitioned by range
into separate partitions for each month,and each of these
partitions is subpartitioned into three separate sales regions.
Note that the partitioning structure is completely mutable:
a user can add new partitions or drop existing partitions or
subpartitions at any point.
Partitioning is important for a number of reasons.First,
the query optimizer is aware of the partitioning structure,
and can analyze predicates to perform partition exclusion:
scanning only a subset of the partitions instead of the entire
table.Second,each partition of a table can have a dierent
storage format,to match the expected workload.A typical
arrangement is to partition by a timestamp eld,and have
older partitions be stored in a highly-compressed append-
only format while newer,\hotter"partitions are stored in a
more update-friendly format to accommodate auditing up-
dates.Third,it enables atomic partition exchange.Rather
than inserting data a row at a time,a user can use ETL or
ELT to stage their data to a temporary table.After the data
is scrubbed and transformed,they can use the ALTER TABLE
...EXCHANGE PARTITION command to bind the temporary
table as a newpartition of an existing table in a quick atomic
operation.This capability makes partitioning particularly
useful for businesses that performbulk data loads on a daily,
weekly,or monthly basis,especially if they drop or archive
older data to keep some xed size\window"of data online
in the warehouse.The same idea also allows users to do
physical migration of tables and storage format modica-
tions in a way that mostly isolates production tables from
loading and transformation overheads.
6.3 MAD Programming
Although MAD design favors quick import and frequent
iteration over careful modeling,it is not intended to re-
ject structured databases per se.As mentioned in Sec-
tion 4,the structured data management features of a DBMS
can be very useful for organizing experimental results,trial
datasets,and experimental work ows.In fact,shops that
use tools like Hadoop typically have DBMSs in addition,
and/or evolve light database systems like Hive.But as we
also note in Section 4,it is advantageous to unify the struc-
tured environment with the analysts'favorite programming
Data analysts come from many walks of life.Some are
experts in SQL,but many are not.Analysts that come
from a scientic or mathematical background are typically
trained in statistical packages like R,SAS,or Matlab.These
are memory-bound,single-machine solutions,but they pro-
vide convenient abstractions for math programming,and ac-
cess to libraries containing hundreds of statistical routines.
Other analysts have facility with traditional programming
languages like Java,Perl,and Python,but typically do not
want to write parallel or I/O-centric code.
The kind of database extensibility pioneered by Postgres [19]
is no longer an exotic DBMS feature { it is a key to modern
data analytics,enabling code to run close to the data.To be
inviting to a variety of programmers,a good DBMS exten-
sibility interface should accommodate multiple languages.
PostgreSQL has become quite powerful in this regard,sup-
porting a wide range of extension languages including R,
Python and Perl.Greenplum takes these interfaces and en-
ables them to run data-parallel on a cluster.This does not
provide automatic parallelism of course:developers must
think through how their code works in a data-parallel envi-
ronment without shared memory,as we did in Section 5.
In addition to work like ours to implement statistical meth-
ods in extensible SQL,there is a groundswell of eort to im-
plement methods with the MapReduce programming paradigm
popularized by Google [4] and Hadoop.From the perspec-
tive of programming language design,MapReduce and mod-
ern SQL are quite similar takes on parallelism:both are
data-parallel programming models for shared-nothing archi-
tectures that provide extension hooks (\upcalls") to inter-
cept individual tuples or sets of tuples within a data ow.
But as a cultural phenomenon,MapReduce has captured the
interest of many developers interested in running large-scale
analyses on Big Data,and is widely viewed as a more at-
tractive programming environment than SQL.A MAD data
warehouse needs to attract these programmers,and allow
them to enjoy the familiarity of MapReduce programming
in a context that both integrates with the rest of the data in
the enterprise,and oers more sophisticated tools for man-
aging data products.
Greenplum approached this challenge by implementing a
MapReduce programming interface whose runtime engine
is the same query executor used for SQL [9].Users write
Map and Reduce functions in familiar languages like Python,
Perl,or R,and connect them up into MapReduce scripts
via a simple conguration le.They can then execute these
scripts via a command line interface that passes the congu-
ration and MapReduce code to the DBMS,returning output
to a congurable location:command line,les,or DBMS
tables.The only required DBMS interaction is the speci-
cation of an IP address for the DBMS,and authentication
credentials (user/password,PGP keys,etc.) Hence develop-
ers who are used to traditional open source tools continue
to use their favorite code editors,source code management,
and shell prompts;they do not need to learn about database
utilities,SQL syntax,schema design,etc.
The Greenplum executor accesses les for MapReduce
jobs via the same Scatter/Gather technique that it uses for
external tables in SQL.In addition,Greenplum MapReduce
scripts interoperate with all the features of the database,and
vice versa.MapReduce scripts can use database tables or
views as their inputs,and/or store their results as database
tables that can be directly accessed via SQL.Hence com-
plex pipelines can evolve that include some stages in SQL,
and some in MapReduce syntax.Execution can be done en-
tirely on demand { running the SQL and MapReduce stages
in a pipeline { or via materialization of steps along the way
either inside or outside the database.Programmers of dier-
ent stripes can interoperate via familiar interfaces:database
tables and views,or MapReduce input streams,incorporat-
ing a variety of languages for the Map and Reduce functions,
and for SQL extension functions.
This kind of interoperability between programming metaphors
is critical for MAD analytics.It attracts analysts { and
hence data { to the warehouse.It provides agility to de-
velopers by facilitating familiar programming interfaces and
enabling interoperability among programming styles.Fi-
nally,it allows analysts to do deep development using the
best tools of the trade,including many domain specic mod-
ules written for the implementation languages.
In experience with a variety of Greenplum customers,we
have found that developers comfortable with both SQL and
MapReduce will choose among them exibly for dierent
tasks.For example,MapReduce has proved more conve-
nient for writing ETL scripts on les where the input or-
der is known and should be exploited in the transformation.
MapReduce also makes it easy to specify transformations
that take one input and produce multiple outputs { this is
also common in ETL settings that\shred"input records
and produce a stream of output tuples with mixed formats.
SQL,surprisingly,has been more convenient than MapRe-
duce for tasks involving graph data like web links and so-
cial networks,since most of the algorithms in that setting
(PageRank,Clustering Coecients,etc.) can be coded com-
pactly as\self-joins"of a link table.
The work in this paper resulted from a fairly quick,it-
erative discussion among data-centric people with varying
job descriptions and training.The process of arriving at the
paper's lessons echoed the lessons themselves.We did not
design a document up front,but instead\got MAD":we
brought many datapoints together,fostered quick iteration
among multiple parties,and tried to dig deeply into details.
As in MAD analysis,we expect to arrive at new questions
and new conclusions as more data is brought to light.A
few of the issues we are currently considering include the
Package management and reuse:In many cases,an
analyst simply wants to string together and parameterize
textbook techniques like linear regression or resampling\o
the shelf".To support this,there is a pressing need { in
both the SQL and MapReduce environments { for a pack-
age management solution and repository akin to the CRAN
repository for R,to enable very easy code reuse.Among
other challenges,this requires standardizing a vocabulary
for objects like vectors,matrices,functions and functionals.
Co-optimizing storage and queries for linear alge-
bra:There are many choices for laying out matrices across
nodes in a cluster [2].We believe that all of them can be
captured in records,with linear algebra methods written
over them in SQL or MapReduce syntax.The next step of
sophistication is for a query optimizer to reason about (a)
multiple,redundant layouts for matrices,and (b) choosing
among equivalent libraries of linear algebra routines that are
customized to the dierent layouts.
Automating physical design for iterative tasks:Ana-
lysts engaged in either ETL/ELT or in core analytics often
take multiple passes over massive data sets in various forms.
They are currently required to think about how to store that
data:leave it external,land it into one of many storage for-
mats,materialize repeated computations,etc.Analysts are
not interested in these questions,they are interested in the
data.It would be useful for a system to (perhaps semi-)
automatically tune these decisions.
Online query processing for MAD analytics:Agility
in the design of analytics depends on how frequently an an-
alyst can iterate.Techniques like Online Aggregation [11]
can be useful to radically speed up this process,but the
literature on the topic needs to be extended substantially
to cover the depth of analysis discussed here.This includes
techniques to provide useful running estimates for more com-
plex computations,including ad-hoc code like MapReduce
programs.It also includes techniques to appropriately pri-
oritize data for processing in a way that captures interesting
data in the tails of distributions.
7.1 When is the Future?
We are not just speculating when we say that MAD ap-
proaches are the future of analytics.For a number of leading
organizations,that future has already arrived,and brings
clear and growing value.But for more conservative organi-
zations,there may not even be a roadmap to get MAD,and
they may have been burned in the past by sta statisticians
or data mining initiatives that failed to gain ground.Those
lessons have to be taken in the context of their time.Return-
ing to Varian's point that opens the paper,the economics
of data are changing exponentially fast.The question is not
whether to get MAD,but how and when.In nearly all envi-
ronments an evolutionary approach makes sense:tradition-
ally conservative data warehousing functions are maintained
even as new skills,practices and people are brought together
to develop MAD approaches.This coexistence of multiple
styles of analysis is itself an example of MAD design.
Thanks to Noelle Sio and David Hubbard of FAN for their
input,and to James Marca of UC Irvine for a x to our
bootstrapping discussion.
[1] T.Barclay et al.Loading databases using data ow
parallelism.SIGMOD Record,23(4),1994.
[2] J.Choi et al.ScaLAPACK:a portable linear algebra
library for distributed memory computers { design
issues and performance.Computer Physics
Computing in Science.
[3] C.-T.Chu et al.Map-Reduce for machine learning on
multicore.In NIPS,pages 281{288,2006.
[4] J.Dean and S.Ghemawat.MapReduce:Simplied
data processing on large clusters.In OSDI,pages
[5] S.Dubner.Hal Varian answers your questions,
February 2008.
[6] M.Franklin,A.Halevy,and D.Maier.From databases
to dataspaces:a new abstraction for information
management.SIGMOD Rec.,34(4),2005.
[7] G.Graefe.Encapsulation of parallelism in the volcano
query processing system.SIGMOD Rec.,19(2),1990.
[8] J Gray et al.Data cube:A relational aggregation
operator generalizing group-by,cross-tab,and
sub-totals.Data Min.Knowl.Discov.,1(1),1997.
[9] Greenplum.A unied engine for RDBMS and
[10] F.R.Hampel et al.Robust Statistics { The Approach
Based on In uence Functions.Wiley,1986.
[11] J.M.Hellerstein,P.J.Haas,and H.J.Wang.Online
aggregation.In ACM SIGMOD,1997.
[12] W.Holland,February 2009.Downloaded from
[13] W.H.Inmon.Building the Data Warehouse.Wiley,
[14] Y.E.Ioannidis et al.Zoo:A desktop experiment
management environment.In VLDB,1996.
[15] A.Kaushik.Web Analytics:An Hour a Day.Sybex,
[16] N.Khoussainova et al.A case for a collaborative
query management system.In CIDR,2009.
[17] K.Lange.Optimization.Springer,2004.
[18] M.T.Roth and P.M.Schwarz.Don't scrap it,wrap
it!A wrapper architecture for legacy data sources.In
[19] M.Stonebraker.Inclusion of new types in relational
data base systems.In ICDE,1986.
[20] M.Stonebraker et al.C-store:a column-oriented
dbms.In VLDB,2005.
[21] M.Stonebraker et al.Requirements for science data
bases and SciDB.In CIDR,2009.
[22] A.S.Szalay et al.Designing and mining
multi-terabyte astronomy archives:the sloan digital
sky survey.SIGMOD Rec.,29(2),2000.
[23] R.Vuduc,J.Demmel,and K.Yelick.Oski:A library
of automatically tuned sparse matrix kernels.In
[24] M.J.Zaki and C.-T.Ho.Large-Scale Parallel Data
[25] Y.Zhang,H.Herodotou,and J.Yang.Riot:
I/O-ecient numerical computing without SQL.In
Enter the password to open this PDF file:
File name:
File size:
Creation Date:
Modification Date:
PDF Producer:
PDF Version:
Page Count:
Preparing document for printing…


text:
Jump to navigation
Latest news, thought leadership, and product information from Sealed Air.
I recently bought a waffle maker online. It was sent to me in the exact same box the manufacturer uses for displaying it on a store shelf, but with a little extra tape. No surprise that it showed up at my house bumped, bruised and broken.
It's that back-to-school time of year again.
Aisles of every store are filled to the brim with pens, notepads and backpacks ready to help students succeed in their pursuit of knowledge. However, not all students have the resources to properly prepare for the school year, which can put them behind in the classroom and cause them to miss critical lessons.
A few years ago, on my way to the office, I was listening to a radio story about the quarterly earnings of a major U.S. retailer. In the story, losses from damaged goods were lumped in the same financial earnings/loss bucket as retail shrinkage and theft.
This type of financial reporting is not surprising, as it has long been true that there's a certain amount of damage loss companies find acceptable. It can be anywhere from two to eight percent, but every business in every industry manages to find its own "acceptable" damage rate.
This time of year is dedicated to remembrance. We remember those who paid the ultimate price, honor those who have served in the armed forces, and appreciate the sacrifices of military spouses and family members. These holidays include Armed Forces Day, Anzac Day and Memorial Day.
While reflecting on the freedoms secured by their efforts, it's important for us to remember to serve those who have served us.
Here's a sentence I never thought I'd write: We need to have a serious conversation about TVs and toilets.
Yes. TVs and toilets.
Ten years ago, buying a toilet or a TV usually meant a trip to a home improvement or electronics store – the kind of place where you wander the aisles staring at different models, mentally weighing the respective features like which one boasts a higher pixel ratio or promises of a quiet flush.
The retail world has been cleverly avoiding the icy grip of the grim reaper for several years now. As consumers continue to shift their whims and wallets with just a few clicks, the reports of retail's demise and potential rebirth keep coming.
In November 2017, Sealed Air announced our commitment to hire veterans into five percent of all open positions in 2018 at our Global Corporate Headquarters. At a recent Sealed Air Veterans Network event, we proudly shared that we're not just meeting our pledge but exceeding it.
In fact, we've hired veterans into an average of 13 percent of open roles in the U.S. since January 2018.
Globally, it is estimated that 35 percent of all consumable fish and seafood is wasted, and distribution and logistical challenges account for nearly 10 percent of supply chain losses.

subset: commoncrawl


text:
So I am looking to possibly book a wedding at Blue Hill Stone Barns but have reservations. It is almost $250pp, with a site fee of $2500. With tax and tip, if you fill the room up, it's $365pp. They raise the price about $10 per year, and the site fee has gone up $1k within 2-3 years. As a result, I'm rethinking this venue. Does anyone have opinions about this place???
Blue Hill Cafe
2427 Route 52, Hopewell Junction, NY 12533


text:
Mom had a restful President's weekend. Her neck is looking fantastic!
It is so amazing to see how quickly her skin is restoring itself. I would have never guessed it would heal so fast! There are just two small spots that are still open and painful. Maybe by the end of the week ..... we can stop slathering her with the Silvadene. She definitely has outstanding genes when it comes to the quality of her skin!
Her saliva is still very thick and continues to cause her difficulties. Apparently this is normal and we hope will pass soon. Because it is so thick and she cannot swallow well, she needs to cough and spit it out. Not much fun at all. It can actually be frightening for her - as she senses herself choking. She is such a "trooper"!
She has good days and slow days, something we can expect for awhile. We will be seeing two of her doctors this week. Dr. Knister on Wednesday morning and Dr. Mirda on Friday. I know they will both be very pleased with her recovery progress.
We await the blooming of the Tulip Tree ..... she looks forward to this each and every year. It's almost as though we know when it blooms, that will be when it is time for her to "step outside". Sunday Mass will be one of her first outings.
We definitely have many prayers of thanks to give!


text:
Publication week blog tour
The Things You Do For Love is on tour through literary blog land in publication week. Follow its progress here...
12/08/16: Life of a Nerdish Mum: WHAT WOULD I TELL MY DAUGHTER ABOUT LIFE AND MOTHERHOOD?
10/08/16 Sue Purkiss book blog: THE GROWTH OF A BOOK
09/08/16 Shaz's Book Blog: WHERE TO WRITE: do you need a room of your own?
08/08/16 Reviewed the Book: WRITERS AND THEIR DOGS
05/08/16 The Writing Garnet: my guest post WRITING WOMEN'S LIVES


text: German cabaret star Lili Marleen inspired a song that was extremely popular with the German soldiers during WW II. This war drama offers a fictionalized account of her story that begins in 1938 while she performs in a Zurich cabaret. It is her boyfriend, a Swiss Jew who also turns out to be a resistance fighter who pens her famous song. She sings it in Germany and it becomes a hit with the German troops. As a result, Hitler himself invites her to perform for him. This does not set well with the songwriter's powerful who, upon learning that Marleen has become a famed singer in Germany, seek to have her barred from Switzerland. This does not stop the songwriter from loving her though and desperate to see her one last time, he sneaks into Berlin for a tryst. Unfortuantely he is arrested and she gets blacklisted. They do not see each other again until after the war. By this time, their lives have changed considerably. This is not considered among the best of Fassbinder's best films.~ Sandra Brennan, All Movie Guide


text:
Thursday, July 9, 2009
Keep Calm, and Carry-Ons…
My father always traveled with his alligator carry-on bag (above in Palm Springs). I don't know where it ended up—probably lost in some high-stakes poker game. I have a couple of his kangaroo leather duffle/gym bags. I rarely use them, because I don't want them to wear out (for sentimental reasons).
This kangaroo leather duffle/gym bag (above) was my father's, probably from the early 1960's. It's black with metallic gold piping. It's all kangaroo leather—which is super durable and lightweight. It looks like it could be a Marc Jacobs bag with the scale of the zipper (below) and it's styling.
If my dad liked something, he would buy multiples of the same item. He always had a bag in this duffle/gym style for traveling to golf tournaments. I ended up with two of these hand-me-down bags in the exact style and colors, but one is a few years older. I wish I had used my dad's strategy of buying multiples of favored items and products, because so many of my favorite things have been discontinued over the years.
I can't afford an alligator bag like my father used. This alligator bag, with a detachable handle, is similar in style to my dad's carry-on...and it's the price of a small car.
Here's a faux (vegan) alligator bag with a similar silhouette, but a considerably lower price tag (and it's on sale too).
Mr. Peacock loves this boxy leather carry-on trolley bag in pebbled leather with a blue lining, and hidden wheels! Most carry-on bags with wheels and pop-up handles aren't very attractive, but this bag has a simple elegance.
It comes in a honey brown...
...and red too!
This limited edition Fred Perry bag is a functional and sharp looking bag…at a reasonable price too. It's made of PVC with contrast piping and has an embedded metal laurel leaf logo on the bottom corner.
This rugged duffle style bag would only get better with age—as the leather gets a bit weathered a beat up a bit.
If your taste is a bit more refined, this luxurious zip-around pebbled leather carry-on bag would fit the bill—not too big, not too small...just right.
Mr. Peacock is very practical when I'm flying. I use an oversize LL Bean tote with a zipper top...
...or my blue canvas (with brown leather trim) zipper top tote that was a sample product on a gig I was working on.
What do you use as a carry-on bag when you're traveling by air?
7 comments:
- Pigtown-Design said...
I have a great tot from Ballard Designs that's jute with leather handles. It's got a zipper and holds a ton. The jute is coated on the inside, so it's water-proof. I also have a Swiss Army black messenger bag that holds a ton.
- Thursday, July 9, 2009 at 6:50:00 AM PDT
- TJB said...
Love the red and honey brown pebbled leather. Oh, to not be in a recession any longer...!
For day/short trips, I use a bag roughly the same size as those, in brown canvas and orange leather. It comfortably holds a toiletry case, an extra set of unmentionables, camera, iPod, etc. For extended vacations, a roomy carry-all in dark grey Taiga leather, which can fit all of the above, plus an extra pair of shoes, a clean change of clothes, etc.
- Friday, July 10, 2009 at 8:04:00 AM PDT
I started an orange leather binge a few years ago because it's my favorite color. I rationalized by saying I'd always be able to spot my bags.
I went with Floto, so it's the Casiana tote every day with the Piana and Venezia for travel.
I didn't order all at once, so my everyday bag is a darker tangerine than the bigger bags.
I hauled everything out for a trip and friends called me a high queen. I choose to believe that's a compliment.
- Friday, July 10, 2009 at 9:32:00 PM PDT
- Mr. Peacock said...
I like to use a tote bag
with a zipper too—
the jute sounds interesting!
I'm always in admiration
of fellow travelers
with colored leather goods.
Orange leather and orange leather trim
have always been appealing to me,
but I haven't taken the plunge
yet and added something
with orange leather...
- Wednesday, July 15, 2009 at 8:35:00 PM PDT
- David Toms said...
If I am going on a short trip I always carry my Pan Am messenger back. Always makes me think of a time when air travel was less hurried and not frought with all the rigmarol it is today
- Monday, July 20, 2009 at 12:19:00 PM PDT
- Blogger said...
Buy Authentic Designer Handbags Online.
- Tuesday, January 10, 2017 at 12:50:00 AM PST
- Blogger said...
Discover how to book international flights for $100 instead of $1,000 using Travel Hacking.
- Saturday, June 3, 2017 at 11:38:00 PM PDT


text:
Today I want to go over Traffic Aid –
Traffic Aid is dished up to you by Andy Black. Andy is renowned in the IM space and I hope this fresh peice of software package will go down well with the community.
<iframe https://www.youtube.com/watch?v=QePRMuDdfE4 width="560″ height="315″ src="https://www.youtube.com/embed/QePRMuDdfE4″ frameborder="0″ allowfullscreen>
Using Traffic Aid, you can have as many as nine people in the video clip conference at any given 1 point of time. You can obtain the movies with just simple url and then change them to audio and appreciate the music.


text:
Munster crush Ospreys
Munster roared into the Heineken Cup semi-finals by crushing out-of-sorts Ospreys 43-9 at Thomond Park.
A lovely touch.
The Irishman was in Paris with his team.
Munster head coach Rob Penney hailed his side as they bounced back from shock opening defeat to Edinburgh as they beat Gloucester 26-10 in the Heineken Cup Pool Six tie at Thomond Park.
Edinburgh skipper Greig Laidlaw kicked 19 points as the Scottish side stunned Munster in a frantic Heineken Cup opener at Murrayfield.
Munster were dumped out of the Heineken Cup at Stade Anoeta as Dimitri Yachvili booted Biarritz to an 18-7 win and into an all-French final in Paris.


text:
At long last, "Immortals" has finally arrived! Director Tarsem Singh's swords-and-sandals epic slashes its way into theaters today (November 11) to deliver one of the greatest visual thrill rides of the fall movie season.
"Immortals" stars Henry Cavill as Theseus, a strong youth who finds himself caught in a war between mortal men and beings from another plane of existence. The vicious King Hyperion (Mickey Rourke) seeks an ancient and terrible power, and only Theseus, who comes to possess the legendary Epirius Bow, can put an end to his reign of terror.
There's little doubt that "Immortals" is the most action-packed movie currently in theaters. For those of you seeking blood, guts and gore, there's no excuse to miss this one. If you need additional reasons to check out "Immortals," we've got five beyond the jump.
Beautiful Violence
If there's one reason to see "Immortals," it's Tarsem Singh's wildly visceral depiction of violence, particularly when gods and titans are thrown into the mix. Action slows to a crawl then speeds up without notice, like a gloriously composed symphony. It's a style that's similar to what we've seen in "300," but it's more than that: the way that the warriors of "Immortals" tear each other to pieces is unlike anything I've ever seen before. The final 20 or so minutes of the film are especially incredible, gifting viewers with some of the most beautiful violence ever committed to the screen.
The Man of Steel
Many actors get their turn in the spotlight throughout "Immortals," but through and through, this is the Henry Cavill show. The future Superman does an excellent job as an action movie lead, his presence captivating and his ability to cleave through the bad guys undeniable. If ever you doubted Cavill's casting in "Man of Steel," his work in "Immortals" should more than convince you that the right choice was made.
Bad to the Bone
Mickey Rourke has been more than a little bit outspoken about his disappointment in how his "Iron Man 2" villain was portrayed. Here, however, Rourke is very clearly having a blast playing Hyperion, a ruthless bad guy who munches on grapes and neuters his prisoners all in the span of seconds. There isn't a lot to like about Hyperion as a human being, no, but Rourke gives you plenty to like about his performance. It's scene-chewing at its finest.
Shirts Need Not Apply
Gents, if you're looking for an excuse to bring a date to "Immortals" this weekend, look no further than the fact that every single guy in this movie is shirtless. Pretty much all the time, too. Lots of muscle for your significant others to feast on, and plenty to make you feel bad about your own physique, too!
Tears (And Blood And Guts And Other Things) In Heaven
From almighty visuals to some memorable turns from Cavill and Rourke, there's a lot to like in "Immortals." But perhaps it's the film's final moments that I'm most enthralled by, largely because the setup for an "Immortals" sequel it provides is just so. Frickin'. Awesome. Seriously, if "Immortals 2" doesn't happen, and if it doesn't pick up immediately where the first movie leaves off, a potentially super-cool ball will have been dropped.
Have you seen "Immortals" yet? Let us know what you think in the comments section and on Twitter!


text:
Say Yum 2005-02-17 - Baby Back Ribs
It's improv night...or some might call it procrastination, or inspiration. Whatever the term, Tony Roma's Ribs are on the menu tonight, per a listener suggestion...thanks Todd! We've also got some Stilton cheese and barleywine ale to whet our appetites inspired from a newspaper article today, and comments from a listener.
Check out the brief slide show of the meal, and our Say Yum team: Devan, and Kris (Coady wasn't included in this one).
Listen to the Say Yum podcast!
SayYum-2005-02-17.mp3 [20.4mb (51:03) 64kbps]
Recipe Follow-Up:
The Stilton cheese on a baguette with the barleywine started off the meal right. Our main course was "stick to your ribs" Say Yum style. About 5 lbs of ribs were boiled with some crushed red pepper flakes, black peppercorns and bay leaves. We only ended up eating about a 1/3 of what we cooked, but the rest will be sealed with our FoodSaver and sent to our chest freezer for a rib craving another night. We broiled the ribs, which had been thoroughly covered in original Tony Roma's sauce, which can be found in some grocery stores or check out the cookbook "Top Secret Restaurant Recipes" by Todd Wilbur for a make your own version. There's also some tasty recipes in that book we might make another night. Use any sauce you want to get the ribs to your liking, but our key to making ribs tender is to make sure the ribs are boiled in water to the point where the ribs are falling off the bone or darn close to it. Some people might not agree with this technique, but it works for us. Thanks for joining us tonight and don't forget to.....say yum!


text:
gla.MAR.ous is a fun, playful, exhilarating fashion line mainly focusing in accessories and adorable party dresses by designer Mary Beirne. The designer gets her inspiration anywhere from classic movies to the pinup and bohemian eras.
Visit gla.MAR.ous for more info and to see the entire collection!
{Shop} {Facebook} {Twitter} {Blog}
Images by gla.MAR.ous


text:
Talking Points: America's Faltering Economy
The most important story in the country today is a faltering economy. And that's the subject of today's Talking Points memo.
The stock market took another huge hit today, down 227 points on the Dow. Treasury Secretary Paul O'Neill's on his way to Europe to assure that community that the American economy is fundamentally strong. Whether that's true or not, I don't know. But I do know this, a couple of months of economic damage, and President Bush will be in major trouble, so he has to do something.
As we told you back in October, Alan Greenspan is a major reason why our economy has contracted. He should be fired, but he can't be because of federal rules, but there are other forces in place as well. Most Americans are in the red. They're still working, but they owe money. We are a consuming society and we spend almost everything we make. The average working man has about $1,000 in savings and more than that in credit card debt.
Talking Points has been also telling you that taxation is also preventing working Americans from stockpiling money. Today, July 6th, is the first day the average American is working for himself. Up until today, everything you've earned will go to the government. From now on, you get to keep your salary. This, of course, is insane, especially since our system provides generally poor public education, sky high prescription drug prices and few amenities for the sick and elderly.
We are not a socialistic society. And if you're going to depend on Uncle Sam, you will get the bare minimum. The fundamental flaw in our system is that no one watches how the tax money is being spent. Thus, billions of dollars are wasted, stolen, or used on ineffective programs. We have proven this time and time again on The Factor.
President Bush is celebrating his 55th birthday today and we wish him a happy day, but come Monday, the president has to begin reassuring the American people that his team has a plan to get the economy moving again. Few on Wall Street or in the corporate world have any confidence right now in Alan Greenspan. So Mr. Bush has to step up. If he doesn't, he will find himself with collapsing approval ratings and his legislative agenda will be in jeopardy because the opposition can smell weakness.
As Bush, the elder, found out the hard way, the economy is the most important issue in America. Somebody better do something fast.
And that's the memo.
Most Ridiculous Item of the Day
Vanity Fair magazine has an article in its August issue called "The Bully Boys of Fox News." Of course, your humble correspondent is one of those boys.
But get this. So is Neil Cavuto, who if you believe this piece should be running the Hells Angels outfit. We'll talk with the guy who wrote the article on Monday. That should be interesting. Until then, it could be ridiculous.
— You can watch Bill O'Reilly's Talking Points weeknights at 8 p.m. ET. And send your comments to: email@example.com


text:
don't know what if any difference this made to the makers of The Mummy, but there was
really an ancient Egyptian woman named Anck-su-Namun...sort of.
Two of my sources give her name as Ankhesenamun and Ankhesnamum. She was, ironically enough, the daughter of Queen Nefertiti! o_0
At age 14, she married King Tutankhamun (Tut to his friends ;-)
Ank. and Tut. did not have any surviving children. The mummies of two infant girls were found in the same tomb as Tutankhamun, and some believe they were their children, possibly stillborn.
After Tut's death, Ankhesnamun was engaged to a Hittite prince, but he was assassinated during the journey to Egypt. She instead married a man named Ay. Ay had presided over Tut's funeral rites, which gave him a tenuous claim to the throne, and by marrying the young pharaoh's widow he cemented that claim. Thus Ay became the next pharaoh of Egypt.
There is some inconclusive evidence that Tut died from a blow to the head. It's tempting to read an ancient murder plot into this, even perhaps one involving Ay himself, but of course such thoughts are all speculation. Still, it's an intriguing possibility that perhaps the historic Ankhesnamun had some of the same troubles as her cinematic counterpart...
Novelist and Egyptologist Christan Jacq says in his Ramses series that Ankhesenamon (yet another spelling) was one of the heretic pharaoh Akhenaton's six daughters. Jacq says she was later executed for her dealings with the Hittites, even though her goal was the protection of Egypt's throne. Akhenaton was an unpopular pharaoh at the time because he urged his people to give up their polytheistic beliefs in favor of worshipping only the god Aton.
Gallery of Images of the Historic Anck-su-Namun


text:
| Home | Contact
Individual Memberships
Existing members, get help started by using the
Computer Support at Your Service!
Tech 4 realtor Individual Membership Plan is ideal for either your home computer or for your business computer, if you are self-employed and need fast, professional computer support throughout the year.
The low rate of $299 of unlimited service for one year is comparable to what other tech support providers charge for a one-time computer repair!
The Individual Membership Plan includes:
Common technology services we can help you with:
Our Online Tech Support Promise:
Make us your very own Personal Tech Support Team!
Email Us


text:
Energy Stew - More about scalar energy power - Energy Stew interview with Tom Paladino - 03.20.15
There might just be a technology that can resolve many health issues.
Tom Paladino is back again to talk about the far reaching aspects of scalar energy and what it means in our lives.
He says he has developed scalar energy machinery based on the work on Nikola Tesla. With this he shows how he can influence the scalar energy field for the betterment of all.
He and I will talk about the nature of scalar energy and how
it works in our lives on a day to day basis. Then we'll look at the way this energy can be controlled and made use of.
I think you'll really want to know about this technology as it may change the life of so many people.


text:
The straight six in the first BMW M5 supersedans was silky smooth and plenty powerful, but it was getting a bit long-in-the-tooth, so it had to go. It was replaced by the 400 hp V8 in the E39, then later with the fire-breathing V10 in the current M5 (above) that pushed horsepower into the stratosphere, particularly when you activated the 'M' button. That engine remains powerful, but with CO2 concerns, fuel economy standards and the fact that the V10 isn't available in much of anything else in BMW's lineup, it seems like the next-gen M5 will probably find its motive power supplied by a V8 instead. If a V8 is good enough for the Audi RS6 and Cadillac CTS-V, perhaps it's good enough for the BMW M5, too.
We saw speculation about a V8 M5 as far back as May, 2008, and more recent speculation that it was going to be a twin-turbo V10 instead, but now Inside Line is reporting that BMW reps have confirmed that a V8 will indeed be in the next M5's engine bay – although they didn't confirm which one. Most likely, it will be the 4.4-liter turbocharged V8 that first appeared in the X5M and X6M. With 555 horsepower and 500 pound-feet of torque, it falls right in the ballpark with other supersedan outputs. That new powerplant will sit in a new cradle as well, with the all-new 5-Series just months away. Around these parts, we're starting to refer to this next generation M5 as the Goldilocks Edition. Six wasn't enough, ten was too much and eight is just right. Thanks for the tip, Derick!
[Source: Inside Line]


text:
The 3M Petrifilm Yeast and Mould Count Plates from Arrow Scientific allow users to identify accurate yeast and mould populations in 3-5 days. The yeast and mould count plates have also been proven as dependable as Antibiotic-supplemented Potato Dextrose Agar (APDA), acidified potato dextrose agar and antibiotic-supplemented plate count agar.
The 3M Petrifilm Yeast and Mould Count Plates deliver accurate results using just three steps:
- Innoculate – testers lift the top film and add sample
- Incubate – space saving design maximises incubator space
- Enumerate – yeast are typically identified by the development of small, blue-green colonies with defined edges and no foci. Molds are indicated by large, variably-colored colonies with diffuse edges. The built-in grid makes counting colonies easy.