Most API providers focus on solving all three of the key challenges for APIs: data gathering, data formatting and data delivery. All three of these functions are critical for the success of an API, however, not all should be solved by the API provider. Rather, the API consumers have a strong, vested interest in the formatting and delivery. As a result, API design should be addressed based on the true separation of concerns between the needs of the API provider and the various API consumers.
This presentation goes into the separation of concerns. It also goes into depth in how Netflix has solved for this problem through a very different approach to API design.
This presentation was given at the following API Meetup in SF:
http://www.meetup.com/API-Meetup/events/171255242/
Keynote at Dockercon Europe Amsterdam Dec 4th, 2014.
Speeding up development with Docker.
Summary of some interesting web scale microservice architectures.
Please send me updates and corrections to the architecture summaries @adrianco
Thanks Adrian
Agile IT Operatinos - Getting to Daily ReleasesLeadingAgile
Getting to Daily Releases with Agile IT Operations. Devin Hedge, Enterprise Transformation Consultant talks to a group at Triagile about the Six Key Areas to focus on when attempting to transform IT Operations with Lean and Agile principles. The talk covers Service Engineering, IT Operations, and the Tier 1 Support/NOC organizations. Kanban, Service Management (ITSM), and what it means to have a DevOps orientation.
What is Agile Service Management? Why Is it Important? ITSM Academy WebinarITSM Academy, Inc.
There has been a lot of buzz lately about the need for IT to be more “agile”. While the term Agile is freely used, there is not a clear definition of what “being agile” actually means, particularly in the context of IT support and service management. This webinar explores the concept of Agile Service Management and how agility helps IT service providers better meet the rapidly changing requirements of business and customers.
Full slide deck for day long discussion of microservices topics. Why use microservices, what options exist and how to migrate to them and address common problems.
Most API providers focus on solving all three of the key challenges for APIs: data gathering, data formatting and data delivery. All three of these functions are critical for the success of an API, however, not all should be solved by the API provider. Rather, the API consumers have a strong, vested interest in the formatting and delivery. As a result, API design should be addressed based on the true separation of concerns between the needs of the API provider and the various API consumers.
This presentation goes into the separation of concerns. It also goes into depth in how Netflix has solved for this problem through a very different approach to API design.
This presentation was given at the following API Meetup in SF:
http://www.meetup.com/API-Meetup/events/171255242/
Keynote at Dockercon Europe Amsterdam Dec 4th, 2014.
Speeding up development with Docker.
Summary of some interesting web scale microservice architectures.
Please send me updates and corrections to the architecture summaries @adrianco
Thanks Adrian
Agile IT Operatinos - Getting to Daily ReleasesLeadingAgile
Getting to Daily Releases with Agile IT Operations. Devin Hedge, Enterprise Transformation Consultant talks to a group at Triagile about the Six Key Areas to focus on when attempting to transform IT Operations with Lean and Agile principles. The talk covers Service Engineering, IT Operations, and the Tier 1 Support/NOC organizations. Kanban, Service Management (ITSM), and what it means to have a DevOps orientation.
What is Agile Service Management? Why Is it Important? ITSM Academy WebinarITSM Academy, Inc.
There has been a lot of buzz lately about the need for IT to be more “agile”. While the term Agile is freely used, there is not a clear definition of what “being agile” actually means, particularly in the context of IT support and service management. This webinar explores the concept of Agile Service Management and how agility helps IT service providers better meet the rapidly changing requirements of business and customers.
Full slide deck for day long discussion of microservices topics. Why use microservices, what options exist and how to migrate to them and address common problems.
Monoliths, Migrations, and MicroservicesRandy Shoup
This talk describes several common challenges of software systems at scale:
* How to break up a monolithic application or a monolithic database into microservices.
* How to approach shared data, joins, and transactions in a microservices ecosystem
Developing a Testing Strategy for DevOps SuccessDevOps.com
To achieve rapid time-to-market, businesses have embraced DevOps, which places a premium on speed and efficiency. But speed is not the only measure of DevOps success. To release better software faster, enterprises must optimize testing strategy and embed a culture of quality within their DevOps processes.
In this webinar, you will learn:
How to transform QA from a bottleneck to a speed enabler
How to integrate quality and increase visibility throughout the SDLC
How to help your VPs and Directors gauge the success of their current quality initiatives
Dynatrace: New Approach to Digital Performance Management - Gartner Symposium...Michael Allen
New cloud stacks, containers, micro-services, automation and DevOps is driving an explosion of application code and infrastructure complexity. It's now nearly impossible to solve the Digital Application Performance Management challenges with traditional tools and approaches. Hear how we are delivering on our vision for Digital performance management, and how the role of digital virtual assistants might transcend into your enterprise. Meet D.A.V.I.S.
Release Planning is a Pain Point in many Agile shops. This is an outline of a process that has worked very well for me over time. I hope you find it useful also.
What Is DevOps? | Introduction To DevOps | DevOps Tools | DevOps Tutorial | D...Edureka!
In this Edureka Devops tutorial, you will learn what is DevOps, and why it is the most efficient software development methodology today. The following topics have been covered in this tutorial:
1. Software Development Challenges
2. How Does DevOps Minimize Challenges?
3. DevOps Tools & Techniques
4. Demand For DevOps Engineers
This is the latest version of the State of the DevSecOps presentation, which was given by Stefan Streichsbier, founder of guardrails.io, as the keynote for the Singapore Computer Society - DevSecOps Seminar in Singapore on the 13th January 2020.
To view recording of this webinar please use below URL:
http://wso2.com/library/webinars/2015/09/event-driven-architecture/
Enterprise systems today are moving towards being dynamic where change has become the norm rather than the exception. Such systems need to be loosely coupled, autonomous, versatile and adaptive. There arises the need to model such systems, and event driven architecture (EDA) is how such systems can be modelled and explained.
This webinar will discuss
The basics of EDA
How it can benefit your enterprise
How the WSO2 product stack complements this architectural pattern
Shift Left Security - The What, Why and HowDevOps.com
The shift left approach in DevOps moves software testing earlier in its lifecycle to prevent defects early in the software delivery process. How can developers use this approach to ensure security? Josh Thorngren, VP of Marketing at Twistlock, will explain what it means to shift left, and share five steps to ensure a successful transition to a shift left approach with DevOps.
Join this webinar to learn:
Best practices in adopting a successful shift to the left
How ‘shifting left’ promotes security
How developers are the new security guards in protecting company information
What is DevOps? | DevOps Introduction | DevOps Tools | DevOps Tutorial For Be...Simplilearn
This presentation on DevOps will help you understand what is DevOps, how DevOps came to being, stages and tools of DevOps, implementation of DevOps, DevOps practices, benefits of DevOps approach and at the end, you will also see a use case of DevOps approach by Etsy. DevOps is a software engineering culture that unifies the development and operations team, under an umbrella of tools to automate every stage. The benefits of DevOps outweigh the potential difficulties. Aligning the two transparency-limited silos ensures that systems are delivered faster, and also reduces risks in production changes through nonfunctional and automated testing, as well as shorter developmental iterations. The DevOps approach automates the service management for the support of operational objectives and improves understanding of the layers in the production environment stack. In turn, this helps prevent and resolve production issues. Now, lets deep dive into these slides and understand what actually DevOps is.
Below topics are explained in this DevOps presentation:
1. How DevOps came to being
2. What is DevOps?
3. Stages and tools of DevOps
4. Implementation of DevOps
5. DevOps practices
6. Use case: DevOps approach by Etsy
7. Benefits of DevOps approach
Simplilearn's DevOps Certification Training Course will prepare you for a career in DevOps, the fast-growing field that bridges the gap between software developers and operations. You’ll become en expert in the principles of continuous development and deployment, automation of configuration management, inter-team collaboration and IT service agility, using modern DevOps tools such as Git, Docker, Jenkins, Puppet and Nagios. DevOps jobs are highly paid and in great demand, so start on your path today.
Why learn DevOps?
Simplilearn’s DevOps training course is designed to help you become a DevOps practitioner and apply the latest in DevOps methodology to automate your software development lifecycle right out of the class. You will master configuration management; continuous integration deployment, delivery and monitoring using DevOps tools such as Git, Docker, Jenkins, Puppet and Nagios in a practical, hands-on and interactive approach.
Who should take this course?
DevOps career opportunities are thriving worldwide. DevOps was featured as one of the 11 best jobs in America for 2017, according to CBS News, and data from Payscale.com shows that DevOps Managers earn as much as $122,234 per year, with DevOps engineers making as much as $151,461. DevOps jobs are the third-highest tech role ranked by employer demand on Indeed.com but have the second-highest talent deficit.
1. This DevOps training course will be of benefit to the following professional roles:
2. Software Developers
3. Technical Project Managers
4. Architects
5. Operations Support
6. Deployment engineers
7. IT managers
8. Development managers
Learn more at https://www.simplilearn.com/
Why does Spotify use a microservices architecture? What are the benefits and challenges we've encountered? How does our organizational model support our architecture?
Video of the talk is posted on YouTube: https://youtu.be/7LGPeBgNFuU
Netflix Edge Engineering Open House Presentations - June 9, 2016Daniel Jacobson
Netflix's Edge Engineering team is responsible for handling all device traffic for to support the user experience, including sign-up, discovery and the triggering of the playback experience. Developing and maintaining this set of massive scale services is no small task and its success is the difference between millions of happy streamers or millions of missed opportunities.
This video captures the presentations delivered at the first ever Edge Engineering Open House at Netflix. This video covers the primary aspects of our charter, including the evolution of our API and Playback services as well as building a robust developer experience for the internal consumers of our APIs.
Monoliths, Migrations, and MicroservicesRandy Shoup
This talk describes several common challenges of software systems at scale:
* How to break up a monolithic application or a monolithic database into microservices.
* How to approach shared data, joins, and transactions in a microservices ecosystem
Developing a Testing Strategy for DevOps SuccessDevOps.com
To achieve rapid time-to-market, businesses have embraced DevOps, which places a premium on speed and efficiency. But speed is not the only measure of DevOps success. To release better software faster, enterprises must optimize testing strategy and embed a culture of quality within their DevOps processes.
In this webinar, you will learn:
How to transform QA from a bottleneck to a speed enabler
How to integrate quality and increase visibility throughout the SDLC
How to help your VPs and Directors gauge the success of their current quality initiatives
Dynatrace: New Approach to Digital Performance Management - Gartner Symposium...Michael Allen
New cloud stacks, containers, micro-services, automation and DevOps is driving an explosion of application code and infrastructure complexity. It's now nearly impossible to solve the Digital Application Performance Management challenges with traditional tools and approaches. Hear how we are delivering on our vision for Digital performance management, and how the role of digital virtual assistants might transcend into your enterprise. Meet D.A.V.I.S.
Release Planning is a Pain Point in many Agile shops. This is an outline of a process that has worked very well for me over time. I hope you find it useful also.
What Is DevOps? | Introduction To DevOps | DevOps Tools | DevOps Tutorial | D...Edureka!
In this Edureka Devops tutorial, you will learn what is DevOps, and why it is the most efficient software development methodology today. The following topics have been covered in this tutorial:
1. Software Development Challenges
2. How Does DevOps Minimize Challenges?
3. DevOps Tools & Techniques
4. Demand For DevOps Engineers
This is the latest version of the State of the DevSecOps presentation, which was given by Stefan Streichsbier, founder of guardrails.io, as the keynote for the Singapore Computer Society - DevSecOps Seminar in Singapore on the 13th January 2020.
To view recording of this webinar please use below URL:
http://wso2.com/library/webinars/2015/09/event-driven-architecture/
Enterprise systems today are moving towards being dynamic where change has become the norm rather than the exception. Such systems need to be loosely coupled, autonomous, versatile and adaptive. There arises the need to model such systems, and event driven architecture (EDA) is how such systems can be modelled and explained.
This webinar will discuss
The basics of EDA
How it can benefit your enterprise
How the WSO2 product stack complements this architectural pattern
Shift Left Security - The What, Why and HowDevOps.com
The shift left approach in DevOps moves software testing earlier in its lifecycle to prevent defects early in the software delivery process. How can developers use this approach to ensure security? Josh Thorngren, VP of Marketing at Twistlock, will explain what it means to shift left, and share five steps to ensure a successful transition to a shift left approach with DevOps.
Join this webinar to learn:
Best practices in adopting a successful shift to the left
How ‘shifting left’ promotes security
How developers are the new security guards in protecting company information
What is DevOps? | DevOps Introduction | DevOps Tools | DevOps Tutorial For Be...Simplilearn
This presentation on DevOps will help you understand what is DevOps, how DevOps came to being, stages and tools of DevOps, implementation of DevOps, DevOps practices, benefits of DevOps approach and at the end, you will also see a use case of DevOps approach by Etsy. DevOps is a software engineering culture that unifies the development and operations team, under an umbrella of tools to automate every stage. The benefits of DevOps outweigh the potential difficulties. Aligning the two transparency-limited silos ensures that systems are delivered faster, and also reduces risks in production changes through nonfunctional and automated testing, as well as shorter developmental iterations. The DevOps approach automates the service management for the support of operational objectives and improves understanding of the layers in the production environment stack. In turn, this helps prevent and resolve production issues. Now, lets deep dive into these slides and understand what actually DevOps is.
Below topics are explained in this DevOps presentation:
1. How DevOps came to being
2. What is DevOps?
3. Stages and tools of DevOps
4. Implementation of DevOps
5. DevOps practices
6. Use case: DevOps approach by Etsy
7. Benefits of DevOps approach
Simplilearn's DevOps Certification Training Course will prepare you for a career in DevOps, the fast-growing field that bridges the gap between software developers and operations. You’ll become en expert in the principles of continuous development and deployment, automation of configuration management, inter-team collaboration and IT service agility, using modern DevOps tools such as Git, Docker, Jenkins, Puppet and Nagios. DevOps jobs are highly paid and in great demand, so start on your path today.
Why learn DevOps?
Simplilearn’s DevOps training course is designed to help you become a DevOps practitioner and apply the latest in DevOps methodology to automate your software development lifecycle right out of the class. You will master configuration management; continuous integration deployment, delivery and monitoring using DevOps tools such as Git, Docker, Jenkins, Puppet and Nagios in a practical, hands-on and interactive approach.
Who should take this course?
DevOps career opportunities are thriving worldwide. DevOps was featured as one of the 11 best jobs in America for 2017, according to CBS News, and data from Payscale.com shows that DevOps Managers earn as much as $122,234 per year, with DevOps engineers making as much as $151,461. DevOps jobs are the third-highest tech role ranked by employer demand on Indeed.com but have the second-highest talent deficit.
1. This DevOps training course will be of benefit to the following professional roles:
2. Software Developers
3. Technical Project Managers
4. Architects
5. Operations Support
6. Deployment engineers
7. IT managers
8. Development managers
Learn more at https://www.simplilearn.com/
Why does Spotify use a microservices architecture? What are the benefits and challenges we've encountered? How does our organizational model support our architecture?
Video of the talk is posted on YouTube: https://youtu.be/7LGPeBgNFuU
Netflix Edge Engineering Open House Presentations - June 9, 2016Daniel Jacobson
Netflix's Edge Engineering team is responsible for handling all device traffic for to support the user experience, including sign-up, discovery and the triggering of the playback experience. Developing and maintaining this set of massive scale services is no small task and its success is the difference between millions of happy streamers or millions of missed opportunities.
This video captures the presentations delivered at the first ever Edge Engineering Open House at Netflix. This video covers the primary aspects of our charter, including the evolution of our API and Playback services as well as building a robust developer experience for the internal consumers of our APIs.
Maintaining the Netflix Front Door - Presentation at Intuit MeetupDaniel Jacobson
This presentation goes into detail on the key principles behind the Netflix API, including design, resiliency, scaling, and deployment. Among other things, I discuss our migration from our REST API to what we call our Experienced-Based API design. It also shares several of our open source efforts such as Zuul, Scryer, Hystrix, RxJava and the Simian Army.
Set Your Content Free! : Case Studies from Netflix and NPRDaniel Jacobson
Last Friday (February 8th), I spoke at the Intelligent Content Conference 2013. When Scott Abel (aka The Content Wrangler) first contacted me to speak at the event, he asked me to speak about my content management and distribution experiences from both NPR and Netflix. The two experiences seemed to him to be an interesting blend for the conference. These are the slides from that presentation.
I have applied comments to every slide in this presentation to include the context that I otherwise provided verbally during the talk.
Revolutions have a common pattern in technology and this is no different for the API space. This presentation discusses that pattern and goes through various API revolutions. It also uses Netflix as an example of how some revolutions evolved and where things may be headed.
What's a good API business model? If you have an API, or you plan to have an open API, or just want to use APIs in your web or mobile app, what models make sense? See 20 different API business models. This comprehensive survey of the gamut of today's options covering anything from paid to getting paid to indirect.
Scaling the Netflix API - From Atlassian Dev DenDaniel Jacobson
The term "scale" for engineering often is used to discuss systems and their ability to grow with the needs of its users. This is clearly an important aspect of scaling, but there are many other areas in which an engineering organization needs to scale to be successful in the long term. This presentation discusses some of those other areas and details how Netflix (and specifically the API team) addresses them.
Versioning schemes and branching models for Continuous Delivery - Continuous ...Pavel Chunyayev
There are many different versioning schemes and branching models. Although there’s no golden standard, some of them suit Continuous Delivery more than others. Both branching and versioning are fundamental to the software development life cycle and
I discussed different methods that communities developed over years along with their advantages and disadvantages.
History and Future of the Netflix API - Mashery Evolution of DistributionDaniel Jacobson
Presentation on the history and future of the Netflix API. This presentation walks through how the API was formed, why it needs a redesign and some of the principles that will be applied in the redesign effort.
This presentation was given at the Mashery Evolution of Distribution session in San Francisco on June 2, 2011.
I gave this presentation to the engineering team at PayPal. This presentation discusses the history and future of the Netflix API. It also goes into API design principles as well as concepts behind system scalability and resiliency.
Automotive Grade APIs – designing for longevityNordic APIs
This is a session given by Henrik Segesten at Nordic APIs 2016 Platform Summit on October 26th, in Stockholm Sweden.
Description:
In the automotive industry, the term “automotive grade” is in common use applied to hardware. It means that the hardware has been tested for longer durability and more extreme conditions than consumer grade hardware. But how does this apply to software and more specifically APIs?
Why should C-Level care about APIs? It's the new economy, stupid.Fabernovel
Why should C-Level care about APIs? It's the new economy, stupid.
In this 2013 version of our study on APIs, we identify how APIs allow companies to effectively pursue the classical triptych of business goals: business development, product development and supply chain management.
Through three new practical cases, discover how 5 companies, coming from the most traditional fields, use private APIs and technologies such as cloud and data in order to renovate their business and to invent new business models.
We studied the case of 5 japanese companies. 5 business cases showing the emerging business trend which consists in working on a private or partner-based API to change business models and find new paths of monetization:
// RETAIL: Seven Eleven, global leader of convenience stores, optimizes its logistics almost in real time by using APIs
// HEALTHCARE: OMRON, creator of connected healthcare devices, monetizes the data provided by its users to forecast epidemics
// INDUSTRY: Honda, mobile constructor, connects its entire car fleet with APIs, to provide intelligent guiding services to their clients
// MEDIA : Cookpad, originally a user-generated recipe website, sells its knowledge in consumer preferences to food-processing industry
// AGRICULTURE: Fujitsu, is getting ready to automate Japan's agriculture, in order to cope with the loss of workforce in Japanese companies
And you, how are you going to reinvent your business thanks to APIs?
La version 2013 de l’étude sur les APIs par FABERNOVEL.
A travers trois nouveaux cas pratiques, découvrez comment 5 entreprises, issues des secteurs les plus traditionnels, utilisent les APIs privées, les technologies cloud et la data pour renouveler leurs business et inventer de nouveaux modèles.
Nous sommes allés au Japon étudier le cas de cinq entreprises :
// RETAIL : Seven Eleven, leader mondial des conveniences stores, optimise sa logistique en temps quasi-réel grâce aux APIs
// SANTE : OMRON, créateur d’appareils connectés pour la santé, monétise les données fournis par ses utilisateurs pour prévoir les épidémies
// INDUSTRY : Honda, constructeur automobile, connecte l’ensemble de sa flotte avec des APIs, pour fournir des services de guidage intelligent à ses clients
// MEDIA : Cookpad, initialement un site de recettes fournis par l’utilisateur, revend son savoir des goûts consommateurs à l’agroalimentaire
// AGRICULTURE : Fujitsu, s’apprête à automatiser l’agriculture Japonaise, afin de faire face à la perte de main-d’oeuvre dans les campagnes japonaises
Et vous, comment allez vous réinventer votre business grâce aux APIs ?
An edge gateway is an essential piece of infrastructure for large scale cloud based services. This presentation details the purpose, benefits and use cases for an edge gateway to provide security, traffic management and cloud cross region resiliency. How a gateway can be used to enhance continuous deployment, and help testing of new service versions and get service insights and more are discussed. Philosophical and architectural approaches to what belongs in a gateway vs what should be in services will be discussed. Real examples of how gateway services, built on top of Netflix's Open source project, Zuul, are used in front of nearly all of Netflix's consumer facing traffic will show how gateway infrastructure is used in real highly available, massive scale services.
Talk about the Netflix API and how it serves as the front door for Netflix device UIs. Topics include: API design, resiliency patterns, scalability, and enabling fast dev/deploy cycles.
Lessons learned on the Azure API Stewardship Journey.pptxapidays
apidays LIVE Singapore 2022: Digitising at scale with APIs
April 20 & 21, 2022
Lessons learned on the Azure API Stewardship Journey
Adrian Hall, Principal Product Manager at Microsoft
------------
Check out our conferences at https://www.apidays.global/
Do you want to sponsor or talk at one of our conferences?
https://apidays.typeform.com/to/ILJeAaV8
Learn more on APIscene, the global media made by the community for the community:
https://www.apiscene.io
Explore the API ecosystem with the API Landscape:
https://apilandscape.apiscene.io/
Deep dive into the API industry with our reports:
https://www.apidays.global/industry-reports/
Subscribe to our global newsletter:
https://apidays.typeform.com/to/i1MPEW
One of the greatest challenges to developing an API is ensuring that your API lasts. After all, you don’t want to have to release and manage multiple versions of your API just because you weren’t expecting users to use it a certain way, or because you didn’t anticipate far enough down the roadmap. In this session, we’ll talk about the challenge of API Longevity, as well as ways to increase your API lifecycle including having a proper mindset, careful design, agile user experience and prototyping, best design practices including hypermedia, and the challenge of maintaining persistence.
apidays LIVE Paris 2021 - Lessons from the API Stewardship Journey in Azure b...apidays
apidays LIVE Paris 2021 - APIs and the Future of Software
December 7, 8 & 9, 2021
Lessons from the API Stewardship Journey in Azure
Ryan Sweet, Principal Architect at Microsoft
This deck is an a joining of ideas from numerous visits to clients around the wound. Here we show the three most common design patterns and explain the pros and cons
APIs used to be a technical implementation detail reserved for developers and architects. In the Web age, APIs make more business sense than ever before. This presentation gives a ring side view of How to Craft Business Strategy around APIs.
API documentation is not a new topic, but we still have undocumented APIs and poorly documented ones. For both public and private APIs, the documentation can make or break the developer experience. Learn how to build awareness, gain adoption, and maintain your API documentation.
How can companies like yours handle Salesforce data through high-performance access or blended reporting? Join us as we describe three ways in which Salesforce data integration can help you achieve lighting-fast business intelligence compatible with your favorite tools.
apidays LIVE New York 2021 - Service API design validation by Uchit Vyas, KPMGapidays
apidays LIVE New York 2021 - API-driven Regulations for Finance, Insurance, and Healthcare
July 28 & 29, 2021
Service API design validation
Uchit Vyas, Associate Director at KPMG
Pain Points In API Development? They’re EverywhereNordic APIs
There’s an inherent tension for organizations doing API development: how to keep both your API developers as well as your infrastructure happy, at the same time. Decoupling front-end and back-end development allows parallel development, and helps keep your front-end, middle-end, and back-end efforts working asynchronously. This speeds progress, but requires far more – and far better – collaboration to be successful. Even an independent developer working with APIs requires good collaboration tools.
In this talk, Abhinav Asthana will provide tips on how to improve in API development using collaboration tools like executable API descriptions, API mock servers, and documentation. He will include specific examples of how companies (such as VMware, Coursera, and AMC Theatres) have used collaboration to attain more agile development, to onboard developers, and to ensure input from all participants/stakeholders.
Main focus of the talk is to communicate some key concepts of designing/implementing APIs based on an enterprise grade API Standards and Guidelines. We will try to handcraft few API recipes(i.e. implementation design) with real-life examples mixed with a live coding session. While working on each recipe, we will delve into the rationale behind design decisions and best practices. We believe that these concepts will help a developer build a comprehensive API solution from scratch.
Are your APIs becoming too complicated and ad hoc? Feeling the need to set up policies for your API? This presentation will give you strategy options for designing and developing your APIs.
DataHero / Eventbrite - API Best PracticesJeff Zabel
If you are considering creating an API or an API Platform, we've provided this guide on what you should consider before creating it. We've done 26 integrations and counting and these are some things we think ever api solution should get right.
Practical Application of API-First in microservices developmentChavdar Baikov
Building complex applications using microservices-based architecture naturally involves a great amount of remote communication based on RESTful APIs. The REST APIs represent the contract of the microservice, with both external and internal stakeholders. Microservices, with poorly designed and unstable APIs, might pose a challenge for
the whole application stack, for both adoption and maintenance.
In this session, we will dig into the benefits of the API-First development approach, for designing stable, clean, and robust microservice APIs. We will showcase a practical example of how API-First development can be streamlined for developing and consuming Spring-based Java microservices, leveraging mostly free and open source technologies.
Maintaining the Front Door to Netflix : The Netflix APIDaniel Jacobson
This presentation was given to the engineering organization at Zendesk. In this presentation, I talk about the challenges that the Netflix API faces in supporting the 1000+ different device types, millions of users, and billions of transactions. The topics range from resiliency, scale, API design, failure injection, continuous delivery, and more.
Many API programs get launched without a clear understanding as to WHY the API should exist. Rather, many are focused on WHAT the API consists of and HOW it should be targeted, implemented and leveraged. This presentation focuses on establishing the need for a clear WHY proposition behind the decision. The HOW and then WHAT will follow from that.
This presentation also uses the history of the Netflix API to demonstrate the power, utility and importance of knowing WHY you are building an API.
The term "scale" for engineering often is used to discuss systems and their ability to grow with the needs of its users. This is clearly an important aspect of scaling, but there are many other areas in which an engineering organization needs to scale to be successful in the long term. This presentation discusses some of those other areas and details how Netflix (and specifically the API team) addresses them.
The term "scale" for engineering often is used to discuss systems and their ability to grow with the needs of its users. This is clearly an important aspect of scaling, but there are many other areas in which an engineering organization needs to scale to be successful in the long term. This presentation discusses some of those other areas and details how Netflix (and specifically the API team) addresses them.
Netflix API: Keynote at Disney Tech ConferenceDaniel Jacobson
Disney held the first in a series of internal technical conferences in Orlando, FL, this one focused entirely on APIs. These slides are from my keynote presentation which kicked off the event. The slides focus on the Netflix API, API design, anti-patterns, technical revolutions, resiliency, scaling, test frameworks and other constructs that support the Netflix infrastructure.
Techniques for Scaling the Netflix API - QCon SFDaniel Jacobson
This presentation was from QCon SF 2011. In these slides I discuss various techniques that we use to scale the API. I also discuss in more detail our effort around redesigning the API.
APIs for Internal Audiences - Netflix - App Dev ConferenceDaniel Jacobson
API programs, typically thought of as a public program to see what public developer communities can build with a company's data, are becoming more and more critical to the success of mobile and device strategies. This presentation takes a look at Netflix's and NPR's strategies that lead to tremendous growth and discusses how Netflix plans to take this internal API strategy to the next level.
This is my presentation from the Business of APIs Conference in SF, held by Mashery (http://www.apiconference.com).
This talk talks briefly about the history of the Netflix API, then goes into three main categories of scaling:
1. Using the cloud to scale in size and internationally
2. Using Webkit to scale application development in parallel to the flexibility afforded by the API
3. Redesigning the API to improve performance and to downscale the infrastructure as the system scales
When viewing these slides, please note that they are almost entirely image-based, so I have added notes for each slide to detail the talking points.
This is a presentation that I gave to ESPN's Digital Media team about the trajectory of the Netflix API. I also discussed Netflix's device implementation strategy and how it enables rapid development and robust A/B testing.
This presentation demonstrates the great successes of the Netflix API to date. After some introspection, however, there is an opportunity to better prepare the API for the future. This presentation also offers a few ideas on how the Netflix API architecture may change over time.
NPR: Digital Distribution Strategy: OSCON2010Daniel Jacobson
When launching the API at OSCON in 2008, NPR targeted four audiences: the open source community; NPR member stations; NPR partners and vendors; and finally our internal developers and product managers. In its short two-year life, the NPR API has grown tremendously, from only a few hundred thousand requests per month to more than 60M. The API, furthermore, has enabled tremendous growth for NPR in the mobile space while facilitating more than 100% growth in total page views in the last year.
NPR's Digital Distribution and Mobile StrategyDaniel Jacobson
The NPR API has been the great enabler to achieve rapid development in the mobile space. That is, because we have our rich and powerful API, our mobile team is free to pursue the development of their mobile products without being encumbered by limited internal development resources. The touch-point between the mobile product and our content is fixed which means the mobile team can focus on design and usability for the specific platform.
These slides demonstrate some of the usage and metrics of the NPR API. In addition to the flow of an NPR story from creation to distribution, I also tried to provide a reasonable sampling of the more popular or interesting implementations.
These slides are from the OpenID UX Summit at Sears in Chicago. We discuss the newly formed Adoption Committee for OpenID, NPR's identity sharing strategy, Sears' OpenID case study, PBS' case study, and the goal towards a federated public media identity.
This presentation shows the same NPR story displayed in a wide range of platforms. The content, through the principles of COPE, is pushed out to all of these destinations through the NPR API. Each destination, meanwhile, uses the appropriate content for that presentation layer.
"Impact of front-end architecture on development cost", Viktor TurskyiFwdays
I have heard many times that architecture is not important for the front-end. Also, many times I have seen how developers implement features on the front-end just following the standard rules for a framework and think that this is enough to successfully launch the project, and then the project fails. How to prevent this and what approach to choose? I have launched dozens of complex projects and during the talk we will analyze which approaches have worked for me and which have not.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at [email protected] to discuss next steps.
Let's dive deeper into the world of ODC! Ricardo Alves (OutSystems) will join us to tell all about the new Data Fabric. After that, Sezen de Bruijn (OutSystems) will get into the details on how to best design a sturdy architecture within ODC.
10. Target Audience Influence
• Team Identity
• Staffing Decisions
• System Architecture
• SLAs
• Development Velocity
• Security Needs
11.
12. Netflix API : Key Responsibilities
2008
• Broker data between internal services and
public developers
• Grow community of public developers
• Optimize design for reusability
15. Private API Public API
< 0.3% of total
API traffic *
* 11 years worth of public API requests = one day of private API requests
16. Netflix API : Key Responsibilities
Today
• Broker data between services and devices
• System resiliency
• Scaling the system
• High velocity development
• Insights
17. The consumers of the API are now
Netflix subscribers
We are now responsible for ensuring
subscribers can stream
21. Primary Responsibilities of APIs
• Data Gathering
– Retrieving the requested data from one or many local
or remote data sources
• Data Formatting
– Preparing a structured payload to the requesting
agent
• Data Delivery
– Delivering the structured payload to the requesting
agent
26. Why do most API providers provide
everything?
• Many APIs have a large set of unknown and
external developers
• Generic API design tends to be easier for
teams closer to the source
• Centralized API functions makes them easier
to support
27. Why do most API providers provide
everything?
• Many APIs have a large set of unknown and
external developers
• Generic API design tends to be easier for
teams closer to the source
• Centralized API functions makes them easier
to support
28. Data Gathering Data Formatting Data Delivery
API Consumer
Don’t care how data is
gathered, as long as it
is gathered
Each consumer cares a
lot about the format
for that specific use
Each consumer cares a
lot about how payload
is delivered
API Provider
Care a lot about how
the data is gathered
Only cares about the
format to the extent it
is easy to support
Only cares that the
delivery method is
easy to support
Separation of Concerns
To be a better provider, the API should address the
separation of concerns of the three core functions
30. Data Gathering Data Formatting Data Delivery
API Consumer
Don’t care how data is
gathered, as long as it
is gathered
Each consumer cares a
lot about the format
for that specific use
Each consumer cares a
lot about how payload
is delivered
API Provider
Care a lot about how
the data is gathered
Only cares about the
format to the extent it
is easy to support
Only cares that the
delivery method is
easy to support
Separation of Concerns
65. Personalization
Service
API
• Build
• Test
• Deploy Service
• Release Lib
Pers.
Lib
• Integrate Lib
• Build
• Test
• Deploy Service
UI Script
Iterations in Hours or Days
Access
Data
66. Personalization
Service
API
• Build
• Test
• Deploy Service
• Release Lib
• Publish to API
Pers.
Lib
UI Script
Iterations in Minutes?
Access
Data
• Integrate Lib
• Build
• Test
• Deploy Service
85. 1. Know Your Audience
2. Separation of Concerns
3. One Size Doesn’t Fit All
4. Be Pragmatic, Not
Dogmatic
5. Embrace Change
1. Act Fast, React Fast
2. Enable Others to Act
Fast, React Fast
3. Internal Developers
Need Engagement Too
4. Failure is Inevitable
5. Scale at All Costs
The lessons that we discuss in these slides fall into two buckets: strategy and implementation.
In some cases, the audience will be a small set of known developers (SSKDs). These developers are generally engineers within your company or one with whom you are partnering.
In other cases, the audience may be a large set of unknown developers. This audience is typically associated with public APIs.
And in some cases, the API will target both audience types.
This is a short list of the things that the target audience will influence.
For Netflix, we started out with a public API, with the audience being a large set of unknown developers. There were no internal use cases at launch.
Based on the target audience of unknown developers, we staffed accordingly. The team was relatively small, with skills around development, evangelism, partnering, testing and documentation.
As streaming became more critical to the company, we started having devices use the API. Our first mistake was that we were probably too late to pivot our architecture based on our change in target audience. At the time, we had many devices call into our REST API, the same one that we used for the unknown developers.
But eventually, the data demonstrated that the architectural change was needed. This chart shows that the private API completely drarfs the public API in terms of requests. The private API does about five billion requests per day while the public API does between one and two million. This disparity clearly demonstrates the need for us to target the API to the small set of known developers – Netflix’s UI engineers – who build the vast majority of the experiences on Netflix devices.
Given the shift in responsibilities, we positioned the team accordingly, hiring for skills mostly around engineering.
And the team size grew by about 6x in the last few years. If the target audience was still the public API, it is likely that the team size would have grown, but less significantly (perhaps 2x) in that time frame.
API consumers care a lot about data formatting and delivery, but each consumer, in such a diverse ecosystem, cares about them differently. For some devices, they may want an XML payload delivered as a complete document, while others may need JSON, protobuffer or some other format, potentially delivered as streamed bits. Because of these diverse needs, we need to separate out the concerns to better enable the consumers to get what they need.
Most companies focus on a small handful of device implementations, most notably Android and iOS devices.
At Netflix, we have more than 1,000 different device types that we support. Across those devices, there is a high degree of variability. As a result, we have seen inefficiencies and problems emerge across our implementations. Those issues also translate into issues with the API interaction.
For example, screen size could significantly affect what the API should deliver to the UI. TVs with bigger screens that can potentially fit more titles and more metadata per title than a mobile phone. Do we need to send all of the extra bits for fields or items that are not needed, requiring the device itself to drop items on the floor? Or can we optimize the deliver of those bits on a per-device basis? Different devices have different controllers as well. Some, like the iPad, allow for fast swipe interactions so the content needs to be there for the entire row. Other devices, like smart TVs or game some game consoles have LRUD controllers, so it at least gives the opportunity to fetch the data as the row gets navigated. And the technical capabilities of the devices will influence the interactions as well. Some have more computing power or memory which will influence how much data you can process on the device vs. how much needs to be gathered in real-time.
We evolved our discussion towards what ultimately became a discussion between resource-based APIs and experience-based APIs.
The original one-size-fits-all API was very resource oriented with granular requests for specific data, delivering specific documents in specific formats.
The interaction model looked basically like this, with (in this example) the PS3 making many calls across the network to the OSFA API. The API ultimately called back to dependent services to get the corresponding data needed to satisfy the requests.
We have decided to pursue an experience-based approach instead. Rather than making many API requests to assemble the PS3 home screen, the PS3 will potentially make a single request to a custom, optimized endpoint.
In an experience-based interaction, the PS3 can potentially make a single request across the network border to a scripting layer (currently Groovy), in this example to provide the data for the PS3 home screen. The call goes to a very specific, custom endpoint for the PS3 or for a shared UI. The Groovy script then interprets what is needed for the PS3 home screen and triggers a series of calls to the Java API running in the same JVM as the Groovy scripts. The Java API is essentially a series of methods that individually know how to gather the corresponding data from the dependent services. The Java API then returns the data to the Groovy script who then formats and delivers the very specific data back to the PS3.
Our original REST API had granular endpoints and generic interaction models. This leads to different versions when significant changes are made. The REST API had three primary version before our move to the experience-based API.
If we persisted in the REST API, we very likely could have continued to add versions while needing to support the old ones. The need to support prior versions stems from older device implementations that may not be able to updated or retired, thus forcing us to maintain these endpoints for a long time (perhaps as long as 10 years).
Our target with the experience-based API was to build an architecture that allowed us to be versionless. Through SSKDs, separation of concerns, abstraction layers, and interaction optimizations, we are able move to a deprecation model.
The primary goal is to limit versioning in the device-to-server interaction. Ideally, we can deprecate effectively in the server interactions as well, but that is sometimes more difficult. Back to our architecture view, the data can now flow from the services into the Java APIs. We expose granular methods (think data elements rather than resources) to the scripting tier. If a method needs to change, we can add a new method and then work closely with the SSKDs to migrate the calling scripts, enabling us to deprecate the old method. If we are not able to move the scripts, we can insulate the devices from the change either in the Java layer or in the scripting tier.
Several years ago, we were deploying changes roughly every two weeks. We would accumulate changes over that time and then drop them into production all at once. Think of it as gathering water in a bucket.
What we found was that our releases were unpredictable, sometimes resulting in outages, broken functionality, or incomplete work. Accordingly, we decided to slow down, changing our release cycles to three weeks. We figured that would give us more time to test our work. In other words, we got a larger bucket.
Over time, however, we learned that the longer release cycle didn’t improve predictability or quality. Instead, it just slowed us down. In response, we moved aggressively towards continuous delivery. Instead of delivering water in buckets, we had a steady stream of water from a hose. This enabled us to have smaller changes, more isolated and testable, pushed to production instead of having bigger releases with more complexity.
This is how code flows through the system. We have multiple canary releases per day. Internal envs are deployed ~8 times/day in 3 AWS regions. Prod deployments happen 2-3 times/week and can be triggered on demand.
This dashboard lets us track the status of our master branch at any time. Builds that fail at any step in the pipeline are stopped from going further.
A quick word on Testing. We follow the ‘Operate what you Build’ model where developers are responsible for shepherding their changes all the way through to production. We provide them with the tools necessary to help them gain confidence in the quality of their code. One such tool is the automated Canary Analyzer.
Canary Analysis is the process wherein a small percent of traffic is routed to the new code and its performance is compared against the old code based on 1000s of metrics.
A detailed report gives further insight into potential problem areas. In this case, our canary gives a score of 87%, which means it is likely not ready for release.
In tandem with canaries, we use Red/Black deployments as well.
The Red/Black process allows us to run production code in one cluster while we spin up the new code in a second one. As the new code proves itself, we can route all traffic to it and eventually shut down the old. It also allows us to have a fast, automated rollback in the even that the new code is seeing problems.
Our architecture enables us to move faster because of the scripting tier. But this also put us in position to help our consuming teams and dependency teams to move faster as well.
Let’s peek under the hood of the API Server. Client teams deploy endpoints dynamically based on their own schedule. Their cycles are completely asynchronous of server deployments. Newly deployed endpoints are live and ready to take traffic within minutes.
Endpoint Activity Dashboard shows recent deployment activity. Rollbacks can be performed in a matter of minutes as well.
Our dependent services provide to us client libraries that get compiled into our JVM upon deployment. These libraries typically expose static interfaces, which means changes to the interfaces require coding and deployments without our contain. Similar to the dynamic endpoints, we also have opportunity to improve the nimbleness and velocity around these libraries.
One such improvement is dependency canaries, where we are evaluating our new code against the dependencies. This is a dashboard the provides insights into these canaries.
Making the interaction with the consumers of the API dynamic has led to increased agility on the UI side. We are also exploring ways to increase the speed of iteration on the dependencies side. The current interaction model uses static domain models and client libraries to handle the data flow through the API. This results in long iteration cycles for even the simplest of use cases. We are actively pursuing an approach where our dependencies will be able to expose new data by using dynamic pass-through model using a Dictionary of key values.
The idea is that this model will avoid the static update cycle on the API end, thereby resulting in shorter iteration cycles. This will require investment in things like safety checks and discoverability of the API. We are instrumenting the API layer to inspect traffic at runtime and provide insights into API usage.
One of the early mistakes that we made in this new architecture was not treating internal developers like we did public developers. We don’t need the same degree of evangelism, but we do need to maintain strong communications with the client teams while providing robust tools and systems to help them be better developers in our system. An example of us being late to this is represented by our endpoint dashboard. One of our teams went from having about 30 scripts to about 500 in a matter of weeks. Each of these scripts are dynamically compiled into the JVM, occupying permgen space. As the script count shot up, we hit limits in our permgen which resulted in an outage. And an outage in our layer means people cannot stream Netflix. Of course, there is nothing like an outage to kickstart new behaviors. As a result, we immediately set up alerts and then focused more heavily on building tools to support the developers.
Included in that effort is comprehensive documentation.
We built an array of tools as well, including this REPL.
And prepared frequent trainings and videos.
Nobody has a 100% SLA, so things will fail
In fact, a few years ago, we have many failures on a routine basis.
Many of those failures were a result of failures in a dependent service that we did a poor job of protecting against. Because we are the last step before delivering content to the customers, we have a unique opportunity to help protect customers from such failures.
Hystrix allows us to be resilient to failure by implementing the bulk-heading and circuit breaker patterns. Hystrix is open source and available at our github repository.
Failure Simulation and Game Day exercises are a key part of the overall story. The Simian army is a fleet of monkeys who are simulate failures and alert us to non-conformities in an automated manner. Chaos Monkey periodically terminates AWS instances in production to see how the system responds to the instance disappearing. Latency Monkey introduces latencies and errors into a service to see how it responds and lets us assess the customer quality of experience. Conformity monkey alerts us to variations in versions of application across regions. The monkeys are also available in our open source github repository
Because of our pivot to the private API and the explosion of devices consuming it, our traffic grew tremendously in a few years (and continues to grow at very fast rates). Scaling our systems to support this growth is absolutely critical to the success of the company. Techniques, such as throttling are not an option because that only serves to limit the interactions from our streaming subscribers. Instead, we need to be able to handle any load that our devices throw at us. This manifests in many ways, but the following is a detail on one of them – instance scaling.
Let’s go back to the traffic chart. The pattern is predictable with higher peaks on the weekends
To offset these limitations, we created Scryer (not yet open sourced, but in production at Netflix). Scryer evaluates needs based on historical data (week over week, month over month metrics), adjusts instance minimums based on algorithms, and relies on Amazon Auto Scaling for unpredicted events
This graph shows that Scryer’s predictions are in line with actual RPS. In production, Scryer allows us to get instances into production prior to the need (which is different than Amazon’s reactive autoscaling engine which triggers the ramp up based on immediate need, only needing to wait until server start-up is complete). Because the instances are there in advance, Scryer smooths out load averages and response times, which in turn improves the customer experience.
This is an example of what Scryer looks like during an outage. When actual traffic dropped because of an outage, the reactive autoscaling engine would have downsized the farm. In this case, Scryer kept the farm sized correctly so that we were able to deal with the traffic spike after the recovery.
As a side benefit (not the initial intent), Scryer also allows us to be more precise with our instance counts, reducing inefficiencies.