PRESTO https://prestodb.io/

WHAT IS PRESTO?

Presto is an open source distributed SQL query engine for running interactive analytic queries against data sources of all sizes ranging from gigabytes to petabytes.

Presto was designed and written from the ground up for interactive analytics and approaches the speed of commercial data warehouses while scaling to the size of organizations like Facebook.

WHAT CAN IT DO?

Presto allows querying data where it lives, including Hive, Cassandra, relational databases or even proprietary data stores. A single Presto query can combine data from multiple sources, allowing for analytics across your entire organization.

Presto is targeted at analysts who expect response times ranging from sub-second to minutes. Presto breaks the false choice between having fast analytics using an expensive commercial solution or using a slow "free" solution that requires excessive hardware.

WHO USES IT?

Facebook uses Presto for interactive queries against several internal data stores, including their 300PB data warehouse. Over 1,000 Facebook employees use Presto daily to run more than 30,000 queries that in total scan over a petabyte each per day.

Leading internet companies including Airbnb and Dropbox are using Presto.

Presto is amazing. Lead engineer Andy Kramolisch got it into production in just a few days. It's an order of magnitude faster than Hive in most our use cases. It reads directly from HDFS, so unlike Redshift, there isn't a lot of ETL before you can use it. It just works.

Christopher Gutierrez, Manager of Online Analytics, Airbnb

We're really excited about Presto. We're planning on using it to quickly gain insight about the different ways our users use Dropbox, as well as diagnosing problems they encounter along the way. In our tests so far it's been rock solid and extremely fast when applied to some of our most important ad hoc use cases.

Fred Wulff, Software Engineer, Dropbox

API developer cookbook

Preparation

Hardware
• own notebooks
• VPN

Software
• Java - version 7, hopefully v8 will come soon
• IntelliJ IDEA or other IDE
• we use IntelliJ Idea, for Spring development we reccomend Ultimate edition (paid)
• Kong
• in a form of Vagrant box
• central OpenAPI team will provide
• VirtualBox, latest version
• Vagrant, latest version
• Git
• XMind, free edition will do (http://www.xmind.net/download/win/)
• Node.js + NPM, version 6 and newer
• all following tools will be installed automatically with base_project.zip
• Dredd - API testing
• Aglio - HTML documentation generation
• Hercule - merging documentation
• Drakov - mocking tool
• Drafter - API Blueprint parser
• (Dredd - API testing, Aglio - HTML documentation generation, Hercule - merging documentation, Drakov - mocking tool, Drafter - API Blueprint parser)
• favorite text editor (API Blueprint), one of:
• Sublime
• Atom
• VIm
• any other, but former have API Blueprint syntax highlighting
• Maven
• Cucumber

Expected knowledge (at least some of those)
• Java (ATL development)
• Spring, Spring-REST - https://spring.io/guides/gs/rest-service/
• REST APIs (in general)
• http://www.restapitutorial.com/index.html
• http://www.vinaysahni.com/best-practices-for-a-pragmatic-restful-api
• OAuth2 - how it works
• https://tools.ietf.org/html/rfc6749
• https://www.digitalocean.com/community/tutorials/an-introduction-to-oauth-2
• DevOps
• Git - https://githowto.com/
• Jenkins - http://www.tutorialspoint.com/jenkins/
• Maven - https://maven.apache.org/guides/getting-started/maven-in-five-minutes.html
• Analysis
• API Blueprint - https://apiblueprint.org/
• MSON - https://apiblueprint.org/documentation/mson/tutorial.html
• Design principles -
• Testing
• Dredd - https://dredd.readthedocs.io/en/latest/
• Cucumber - https://cucumber.io/
• Kong API gateway - basic principles - https://getkong.org
• Open API architecture -

API Development Tools

Component	Proposed Tool(s)	Comments	Link(s)	Roles	Next steps
Source Code Repository	GitLab	A distributed code versioning tool based on GIT with support of pull requests and code review will be set up.	git.ibts.net	Everyone from OpenAPI community should have r/w access to openapi repositories	IDM account (OFFICE) is sufficient for RO access. RW access to OpenAPI project to be done manually by project administrator.
			see also Versioning and branches
		openapi/VNi-api - project for OpenAPI design & interfaces implementation
		openapi/VNi-impl - project for OpenAPI ATL implemetnation
Build Tool	Jenkins	An automated build and deployment tool.	hudson.ibts.net (current)	Everyone from OpenAPI community should have r/w access to openapi build jobs	IDM account (OFFICE) for log in.
			jenkins.com.vn (since Feb 2017)		Create new security group in office domain, which will be mapped to our project OT role. Based on this role community will have access only to openapi part of jenkins. Specific access to be prepared on jenkins by admin - Martin Kulich. We need to define what we need.
		Local instance is recommended to speed things up and save some network traffic.
Component Repository	Nexus	A SW component repository for distribution of ready Open API components (such as the API GW, Credential Store, Developer's Portal).	nexus.com.vn	Everyone from OpenAPI community should have read access, write access only for Jenkins jobs.	IDM account is sufficient (for RO access log in is not needed).
		Local instance is recommended to speed things up and save some network traffic.
Code Quality	Sonar	An automated code quality check tool.	sonar.com.vn	tbd	VNG account is sufficient.
Container / VM	Docker	Component deployment tool	?	tbd
Issue Tracking	JIRA	Separate API project in JIRA issue tracker that supports definition and control of:	https://jira.ibts.net/jira/browse/API	Everyone from OpenAPI community has r/w access to API project.	RV_APP_JIRA_GENERAL, API project, Executive user
		Business Service – high-level business task (one item on the Open API - Business Service Stack)
		EPIC, Story, Task (SubTask) – agile board for the planned local and central development
		Bug – unplanned issue
		Public read acces to all VNI employees (role RV_APP_JIRA_GENERAL). All Open API team members (including Country teams) will have read/write privilege assigned by Benda Jan (role Executive user).
Project Documents	wiki	This wiki	API Home	Everyone from OpenAPI community has RW access to API wiki.	RV_APP_WIKI01_API_RW
		Each team member should have read access to it at least.
		In case when edit privilege is required, contact Benda Jan (add role RV_APP_WIKI01_API_RW).
File Share	VN DepositBox	A web-based solution for exchange of large files (e.g. Vagrant boxes).	https://VN-depositbox.ibts.net/	Everyone with an VNG account should be able to log in and post/receive files. Anyone outside of VNG can drop a file for someone inside VNG.	VNG account is sufficient.
		VN-depositbox is a service to make it easy for you to move files, including large files up to 169.6 GB, in and out of EmbedIT.
Static API Documentation	public web	A www server hosting the public documentation (should be part of the Developer's portal?).	?

OpenAPI Specification format comparison

Category	Title	Swagger	RAML	API Blue Print
Summary	Pros	High adoption rate, good code-level tooling, large and active community of users and supporters, greater support for multiple languages	Good online design tools and mature supporting infrastructure, seemingly useful advanced constructs, human readable format, hight industry backing	Good online design tools, good community involvement, easy to understand, simple to write
Summary	Cons	Bottom-up, lacks advanced constructs for metadata	Very new to the game, low adoption, lacks code-level tooling, still unproven long-term	New to the game, low adoption, lacks advanced constructs and code-level tooling, complex installation
Basic	Primary Sponsor	SmartBear	MULESOFT	APIARY
Basic	Format	JSON	YAML	MARKDOWN
Basic	Initiatives towards Industry Standard	Open API Initiative (OAI) - Has announced an open governance model around the Swagger Specification under the Linux Foundation.	Not Availaible	Not Availaible
Basic	Open Source Inititives	The Swagger page claims  itself to be 100% open source software. Starting January 1st 2016 the Swagger Specification has been donated to to the Open API Initiative (OAI) and has been renamed to the OpenAPI Specification.	RAML is backed by a large open source community providing hundreds of pre-built, customizable tools.	API Blueprint is completely open sourced under the MIT license.
Basic	Industry Backing	Reverb, 3Scale, Apigee	Mulesoft, SOA, AngularJS, Intuit, Box, PayPal, & Programmable Web	Apiary
Basic	Current Version	2	1	1A9
Basic	Workgroup	Yes	Yes	No
Basic	Initial commit	July 2011	September 2013	April 2013
Basic	API Design Approach	Bottom-up	Top-down	Top-down
Community	Community Based	By far has the largest comnmunity and has been around since 2011.	Has gained lot of traction of late in the enterprises. RAML is backed by a large open source community providing hundreds of pre-built, customizable tools for all your RESTful API needs	Is up and coming
Community	Stackoverflow questions	3181	288	696
Community	GitHub Stars	3773	391	3002
Community	GitHub contributors	114	26	38
Community	GitHub whatchers	276	154	147
Community	GitHub forks	2196	118	872
Community	Google search (name + “rest”)	733K	85K	350K
REST Modeling	Stub generators	Yes	Yes	Yes (Limited Languages).
REST Modeling	Resources	X (“api”)	X	X
REST Modeling	Methods/ Actions	X (“operation”)	X (“method”)	X (“action”)
REST Modeling	Query Parameters	X	X	X
REST Modeling	Path/URL Parameters	X	X	X
REST Modeling	Header Parameters	X	X	X
REST Modeling	Representations (status codes, mime-types)	X	X	X
REST Modeling	Documentation	X	X	X
REST Modeling	Authentication	Basic, API-Key, OAuth 2	Basic, Digest, Oauth, 1&2
REST Modeling	Representation Metadata	JSON Schema (subset)	(inline/ external)	(inline)
REST Modeling	Nested Resources		X	X
REST Modeling	Composition/ Inheritance		Traits, Resource Types	Resource Models
REST Modeling	File inclusions		X
REST Modeling	API Version metadata	X	X
REST Modeling	Sample Representations		X	X
Tooling	Live Documentation Demo	Yes	Yes	Yes
Tooling	Authoring	(3rd party)	API-Designer	apiary.io
Tooling	Ad-hoc testing	Swagger-UI	API-Console	apiary.io
Tooling	API Console	Every HTTP Action Verb is separate	Grouped by endpoint with drill down into verbs	Unknown
Tooling	API Builder	Tools for most popular languages	MuleStudio APIKit (Visual, drag and drop), Osprey (Node)	Node & .Net bindings
Tooling	Documentation Builder	Single page documentation builder	Single or multiple page documentation builder (PHP only)	Single or multiple page documentation builder
Tooling	Mocking	(3rd party)	X	X
Tooling	Server Code	Java, scala, ruby, node.js	Java	(3rd party)
Tooling	Generate from code	Java, 3rd party (go, python)
Tooling	Language Support	Clojure, Go, JS, Java, .Net, Node, PHP, Python, Ruby, Scala	JS, Java, Node, PHP, Python, Ruby	Node, .Net
Tooling	Validation	X	X (java)	X
Tooling	Parsing	Java, js	Java, js	C++ (nodejs, c#)
Tooling	Easy of Getting Started	Very easy, everything in Github repos	Very easy, everything in Github repos	Installation required