jkebeck / Workshop Donkeytracker
Labels
Projects that are alternatives of or similar to Workshop Donkeytracker
DonkeyTracker Workshop
In this self-guided workshop you will build an end-to-end serverless tracking application for your mobile devices powered by Amazon Web Services (AWS).
The beauty of serveless computing is, that it allows you to focus entirely on your application rather than managing servers and scaling the environment. It is often also more cost efficient since you avoid overprovisioning and always have just the right capacity. This workshop is build around technologies such as DynamoDB, Lambda, API Gateway, Simple Storage Service (S3), CloudFront, Route 53 and Cognito.
This initial architecture will grow and change if you follow the optional steps from 11 onwards just like a real project. By the end of step 17 it will look like this:
Requirements
- An AWS Account. If you don't have one yet, head over to Appendix 3 to learn how to get one and configure the initial Identity and Access Management. You will also need an AWS Access Key ID and AWS Secret Access Key for some of the testing from the AWS CLI. Appendix 3 also explains how to get the key.
- (Optional) A mobile device with an application for Multi-Factor Authentication (MFA) such as Google Authenticator (Google Play or iTunes)
- A computer with access to the internet and where you have the rights to install software or that has already the following dependencies installed.
- A Code Editor. I prefer Visual Studio Code and use the following extensions for this workshop:
- Markdown All in One for the documentation in Markdown
- Cordova Tools for the mobile application development
- Postman or a similar tool to craft and test HTTP POST requests.
- The mobile application is develped in Apache Cordova, a framework for the development of cross-platform, hybrid applications that translates HTML5 and JavaScript to native code. While you write the code only once and can deploy anywhere, the requirements for the development environment will vary depending on our development machine and the platforms you develop for. If you build on a Mac and deploy to iOS you will find more details on how to set up the machine in appendix 4. For other development machines and platforms check out the Apache Cordova Get Started and the platform specific guides.
- A Mapbox account for the mapping part.
Step 1: Persisting Data in DynamoDB Table
AWS provides many options to store and query data. For the type of data that we want to store in our tracking application, a database is probably most appropriate and AWS provides severeal managed and unmanaged services. While the recently announced Amazon Aurora Serverless might be an option in the future, we'll pick for now the already proven NoSQL database Amazon DynamoDB.
Head over to step 1 to dive right in.
Step 2: Writing Data to DynamoDB Table with Lambda
AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume - there is no charge when your code is not running. Lambda truely simplifies the development as long as your code meets the requirements, i.e. if you're developing APIs that don't require more than 5 min of execution time or 3 GB of memory it is pefect to get started quickly, avoid the management of servers and save cost. It is outside the scope of this workshop but for more complex scenarios where you may want to orchestrate multiple Lambda functions, have a look at AWS Step Functions.
For now head over to step 2 to create your Lambda function.
Step 3: Exposing your Lambda Function through the API Gateway
Amazon API Gateway is a service that enables developers to create, publish, maintain, monitor, and secure APIs at scale. You can create APIs that access AWS or other web services, as well as data stored in the AWS Cloud. There are several ways to athorize access to the API, including a simple API Key system with usage plans that allow you to throttle API requests. For this workshop you will take advantage of these usage plans to authorize access to the API.
So, head over to step 3 to set up the API Gateway and create a method that triggers the Lambda function, you created in the pevious step.
Step 4: The Mobile Device Application
As mentioned above, you will develop this mobile application using Apache Cordova, a framework for building cross-platform, hybrid application that translates HTML 5 and JavaScript into native code to access certain device capacilities such as the GPS (or similar) chip that determines the location. This mobile application shall
- determine the device location, speed, heading, altitude and a few other things,
- send the data to your API Gateway, so that you can trigger the Lambda function and put it to the DynamoDB table
- and do so even if it runs in the background.
As mentioned in the requirements, your machine should be prepared for the development of Apache Cordova applications and if you build on a Mac and deploy to iOS you will find more details on how to set up the machine in appendix 4.
If you are ready to work with Apache Cordova, head over to step 4.
Step 5: Adding a Map to the Mobile Device Application
There are several good mapping services you can choose from when adding an interactive map to an Apache Cordova application: Mapbox, Bing Maps, Google Maps, HERE or TomTom to name a few. Of course you can also use open data such as OpenStreetMap to run your own mapping services. In fact, if you only want to serve maps, as we do in this workshop, it isn't all that difficult and may in fact be more cost efficient than a 3rd party mapping services. Perhaps, we look at this in an optional step later on. For now you'll work with Mapbox, a great service from a great company with lots of open source projects here on GitHub that has done some phenomenal work leveraging and contributing to OpenStreetMap and other related open source and open data projects.
Head over to step 5 to integrate Mapbox into your Cordova application.
Step 6: Website Hosting with S3
Amazon S3 provides object storage with 11 9's of durability and can be easily configured to serve static websites. In the spirit of serverless application that's an ideal scenario for us.
Navigate to step 6 to get started.
Step 7: User Management with Cognito
For the authentication and authorization of access to your website you will use Amazon Cognito. Cognito scales to millions of users, and supports sign-in with social identity providers such as Facebook, Google, and Amazon as well as enterprise identity providers via SAML 2.0. For this workshop you will work with Cognito User Pools rather than federating with identity providers.
Navigate to step 7 to dive right in.
Step 8: Adding Authentication and Authorization to your Website
In the previous 2 steps you have created your website and configured a Cognito User Pool to manage authentication and authorization. It is now time to configure your website to make use of the user pool.
Navigate to step 8 to modify your website.
Step 9: Create an API for your Website
With the structure and user management for your website in place, the final piece is to create an API that retrieves data from your DynamoDB table and displays it on the map. Creating the API is very similar to what you have already done in step 2 and 3. Therefore we'll go through this a bit quicker.
Head over to step 9 to dive right in.
Step 10: Connect your Website to your New API
The code for your website is already prepared to call your newly created API but you still need to make a modification to config.js to provide the Invoke URL for your API Gateway and a key. In step 10 you will make this final change.
Step 11 (optional): Going to the Edge with CloudFront
Amazon CloudFront is a content delivery network (CDN) that delivers data, applications and APIs to end users worldwide with low latency and high transfer speeds. CloudFront is integrated with AWS infrastructure and services and takes advantage of AWS Shield for DDoS mitigation. The latter is quite relevant not just to keep your website available but also to prevent excessive cost that such a DDoS attack could cause.
Note: The API Gateway is by default already integrated with CloudFront.
Head over to step 11 to provide a global presence, faster response times and DDoS protection to your website.
Step 12 (optional): Set up a Domain
Amazon Route 53 is a highly available and scalable Domain Name System (DNS). Aside from registering domains and resolving domain names to IP v4 and v6 addresses inside and outside of AWS, it can also be used to perform health checks, fail-over between entire regions or to provide different types of routing such as latency-, weighted round robin- or geographic proximity-based routing.
We won't cover region failover and different types of routing in this workshop but head over to step 12 to register your domain and point it to your CloudFront distribution.
Step 13 (optional): WWW Domain Redirect
To some extend it is a matter of taste if you have a domain with or without the prefix www - or both. If you want to have them both, head over to step 13 next.
Step 14 (optional): Adding SSL Support
Sending sensitive data such as your user ID or a password over the open internet is a recipe for disaster. In this final step you will create a SSL/TLS certificate using the AWS Certificate Manager to secure your website.
Step 15 (optional): Customizing the User Sign-Up to create a Device Lookup Table
As you start using the application, you'll discover many things you'd like to improve and come up with new ideas you want to implement as well. For example:
- Device IDs are difficult to remember and inconvenient to type. You may want to create a more easy to remember device name and create a look-up table for it
- You may want to look at a date picker for the date
- You may want to replace the default domain names for the APIs with a custom domain name
- You may want to analyze the tracks to detect trips with start- and end-time, human readable name for the start- and end-location, mileage and more
- You may find that you create a bunch of check-ins and want to set up a CI/CD (Continuous Integration / Continuous Deployment) pipeline
In the next steps we go through a few of those and introduce additional features and technologoes along the way. In step 15 we look at the first 2 bullet points. To create a device table you will be Customizing Cognito User Pool Workflows by Using AWS Lambda Triggers.
Step 16 (optional): Set up a Custom Domain name for your API Endpoints
During this workshop you have already configured a custom domain name for your web application but the APIs are still using less memorable automatically generated URLs such as https://api-id.execute-api.region.amazonaws.com/stage. In this step we will change this and set up a custom domain for your APIs.
Step 17 (optional): Using Step-Functions to Orchestrate a Trip Detection Workflow
One of the things you identified above as a significant feature to add was the trip detection. You want to get from individual GPS traces that are streaming in location by location to a point where you can identify trips with start- and end-time, mileage etc. This seems like a good time to also introduce routine tasks such as backup and cleaning up the raw trace data. These tasks can be easily implemented in just one Lambda function but since there is a few things that should be orchestrated in a certain sequence and triggered in regular intervals this is a good time to think about de-coupling of tasks and implementation of workflows. While there are several options such as coding it yourself or Amazon Simple Workflow Service (SWF), a great vehicle for serverless applications are AWS Step Functions. Head over to step 17 to find a simple implementation.