Web as a data resource

Get me my Data !Use the web as a data source

Hack U @ Georgia Tech - March 8 – 12 20112

Subramanyan MuraliyahooMail Engineer Hacker, Photographer, Traveler @rmsguhan

Putting something together in 24 hours can be a real rush !5

Internet is a data exchange medium

Need innovative ways to make the data into information that is useful 7

Any resource on the internet is a source of data … 8

… that means that there is a lot of data 9

Loads and loads of data and data services 10

Data that is complex and in many formats 11

Navigating through all the complexity can be hard !#$%&12

How to solve the data problem ?13

http://pipes.yahoo.comsaves the day!14

Graphical interface to play with data !15

Predefined presentations !RSS, JSON, Map, badge17

Pipes is a visual tool, where is my command line ??18

Common data access language across the webdeveloper.yahoo.com/yql20

SQL like syntaxStandard and easy to use21

select {what} from {service} where {condition} in ({sub-query} )22

Several services can be chained using in ()23

Lots of Integrations and predefined tablesanyone can be part of the interface25

Fetching data from a resource has never been simpler !26

http://query.yahooapis.com/v1/ public/yql?q={query} &format={format} 27

Irrespective of the source, output can be standardized …28

… thus giving you a solid & consistent data foundation29

Input can be XML, HTML, CSV, micro-formats, Atom feeds, JSON, RSS30

Mix and match YQL and Pipes ?32

Pipes provides you with processing capabilities33

Interlinking YQL and Pipes = fewer lines of code35

select * from google.translate where q in (select text from twitter.search where q="georgia tech") and target='fr';

Fire off multiple queries at once

select * from query.multi where queries=‘{query1};{query2}’;

Create your own tables with Open tables www.datatables.org/

More Related Content