Monday, March 24, 2008

JAQL: A Query Language for Jason

·         Kevin Beyer from IBM (did the DB2 Xquery implementation)

·         Why use JSON?

o   Want complete entities in one place (non-normalized)

o   Want evolvable schema

o   Want standards support

o   Didn’t want a DOC markup language (XML)

·         Designed for JSON data

·         Functional query language (few side effects)

·         Core operators: iteration, grouping, joining, combining, sorting, projection, constructors (arrays, records, values), unesting, ..

·         Operates on anything that is JSON format or can be transformed to JSON and produces JSON or any format that can be transformed from JSON.

·         Planning to

o   add indexing support   

o   Open source next summer

o   Adding schema and integrity support

 

DryadLINQ: Michael Isard (Msft Research)

·         Implementation performance:

o   Rather than temp between every stage, join them together and stream

o   Makes failure recovery more difficult but it’s a good trade off

·         Join and split can be done with Map/Reduce but ugly to program and hard to avoid performance penalty

·         Dryad is more general than Map/Reduce and addresses the above two issues

o   Implements a uniform state machine for scheduling and fault tolerance

·         LINQ addresses the programming model and makes it more access able

·         Dryad supports changing the resource allocation (number of servers used) dynamically during job execution

·         Generally, Map/Reduce is complex so front-ends are being built to make it easier: e.g. PIG & Sawzall

·         Linq: General purpose data-paralle programming contructs

·         LINQ+C# provides parsing, thype-checking, & is a lazy evaluator

o   It builds an expression tree and materializes data only when requested

·         PLINQ: supports parallelizing LINQ queries over many cores

·         Lots of interest in seeing this code out there in open source and interest in the community to building upon it.  Some comments very positive about how far along the work is matched with more negative comments on this being closed rather than open source available for other to innovate upon.

 

James Hamilton, Windows Live Platform Services
Bldg RedW-D/2072, One Microsoft Way, Redmond, Washington, 98052
W:+1(425)703-9972 | C:+1(206)910-4692 | H:+1(206)201-1859 |
JamesRH@microsoft.com

H:mvdirona.com | W:research.microsoft.com/~jamesrh  | blog:http://perspectives.mvdirona.com

 

Monday, March 24, 2008 11:47:35 PM (Pacific Standard Time, UTC-08:00)  #    Comments [0] - Trackback
Services
Comments are closed.

Disclaimer: The opinions expressed here are my own and do not necessarily represent those of current or past employers.

Archive
<March 2008>
SunMonTueWedThuFriSat
2425262728291
2345678
9101112131415
16171819202122
23242526272829
303112345

Categories
This Blog
Member Login
All Content © 2014, James Hamilton
Theme created by Christoph De Baene / Modified 2007.10.28 by James Hamilton