Serverless: 15% slower and eight times more expensive
I recently decided to experiment with the API on our CardGames.io website and try the Serverless framework. Over the past few years, it has become a hot topic in the world of technology, and I procrastinated I wanted to keep my technical skills up to date and try something new. Therefore, I decided to spend several hours studying Serverless and see if there is any sense in such an API placement.
CardGames.io runs on AWS. All html pages, CSS, JavaScript and images are stored in S3. We have a C # API hosted on Elastic Beanstalk, it runs on Linux servers running .NET Core with Docker. Finally, we use CloudFront CDN before statics on S3 and API. Below is the EC2 score for August 2019. We have several other instances, but the APIs work on m1.small (yes, t2.small probably works better) with classic load balancing. If you summarize the highlighted in red, then it turns out $ 164.21 a month, not bad. I even included EBS there, because I’m not sure how to break these costs, because we have several projects on EC2.
AWS Account for Elastic Beanstalk
We have two environments with 1-3 instances in each: active and inactive, because the .NET Core deployment in Docker on AWS takes several minutes, so we do it in an inactive environment, and then switch the CNAME records to the recently deployed one. Slow deployment was one of the reasons I wanted to try something new. We have several servers with node.js applications on Beanstalk - and they are deployed in seconds. I wanted it to be the same for the API.
I studied a very good ASP.NET Web API hosting tutorial with the Serverless framework and found that you only need to add a simple configuration file, one dependency and a small startup class to an existing API project. Deploy took about twenty seconds - much better than in Beanstalk. I think thanks to the built-in support for .NET Core in Lambda (however, only 2.2), while in Beanstalk it is only supported if you use independent docker. In any case, I was happy without thinking about auto-scaling groups, max instances and the like.
Serverless on AWS is Lambda, which really hosts your features, and the Front Gateway API, which allows you to add things like speed limits, API keys, and more. I hosted the Lambda features in the us-west-2 region, where the Beanstalk servers are. Then I configured the CloudFront instance to route requests from one of our games to the new serverless configuration, and the other to the old Beanstalk configuration. Then he launched a simple test on two URLs: one Serverless, the other Beanstalk. Both URLs invoked the same API, storing the same event in the database. I ran 100 queries for each, here are the results:
Performance comparison
In multiple runs, Serverless configuration was 15% slower (if anything, I ran it from Iceland, hence the big ping). The speed was a disappointment, but it remained quite high - I realized that there is some overhead for the API Gateway front in front of our API: there are too many functions, even if we do not use them. So, sad, but not critical!
Honestly, at first I did not think about prices. I just decided that paying for actual use would probably be cheaper than paying for instances 24/7. Therefore, it allowed the new serverless installation to work for several days, and then began to check accounts. Oops! The bill for Lambda + API Gateway has already exceeded one hundred dollars! At first I started messing with the Lambda setup, allocating less memory for it to save, but when I looked at what was going on well, it became obvious that the problem was with the gateway. Here are the API Gateway rates:
Gateway API Tariffs
Our API accepts about 10 million requests per day. This is approximately $ 35 per gateway only each day . In addition, Lambda costs about $ 10 per day, although this can be reduced by allocating less memory. Together it turns out about $ 45 per day, or $ 1350 per month , versus $ 164 per month on Elastic Beanstalk. Eight times more expensive ! I like new technologies and a quick deployment, but I'm not going to pay an extra $ 1200 per month for this. Back to Beanstalk!
Probably, I should have familiarized myself with the prices in advance and made some mathematical calculations! But of course, then I would have to do real work, and not learn valuable technical skills! I am sure that in some situations, the Gateway and Lambda APIs are better than the Elastic Beanstalk. It’s just not our case. Perhaps if you use API keys, speed limits and other Gateway API functions, then it makes sense to pay $ 3.50 per million requests. We would be better off just putting a normal load balancer in front of Lambda. As far as I know, this is not possible, Gateway API is required for http access to Lambda.
But even if we just paid for Lambda, at $ 10 a day it would have turned out $ 300 a month instead of $ 164. We have many queries, but each query does very little: basically, one DB call per query. Perhaps heavier queries that use more computing time are better suited for Lambda, where you pay for 100 ms of computing time. Below is a report for one request. As you can see, we use 3.50 ms of computing time, but the invoice is set for 100 ms, this is not a weak rounding.
Lambda report for one request
Finally, I don’t criticize Gateway, Lambda or Serverless APIs at all. Just showing that for some workloads they are much more expensive than the boring old EC2 and Elastic Beanstalk. On what we will remain. It is also likely that there is a much better or more efficient way to configure the system, I am by no means an AWS expert, so if you see any flagrant errors, be sure to indicate in the comments.
Current configuration
CardGames.io runs on AWS. All html pages, CSS, JavaScript and images are stored in S3. We have a C # API hosted on Elastic Beanstalk, it runs on Linux servers running .NET Core with Docker. Finally, we use CloudFront CDN before statics on S3 and API. Below is the EC2 score for August 2019. We have several other instances, but the APIs work on m1.small (yes, t2.small probably works better) with classic load balancing. If you summarize the highlighted in red, then it turns out $ 164.21 a month, not bad. I even included EBS there, because I’m not sure how to break these costs, because we have several projects on EC2.
AWS Account for Elastic Beanstalk
We have two environments with 1-3 instances in each: active and inactive, because the .NET Core deployment in Docker on AWS takes several minutes, so we do it in an inactive environment, and then switch the CNAME records to the recently deployed one. Slow deployment was one of the reasons I wanted to try something new. We have several servers with node.js applications on Beanstalk - and they are deployed in seconds. I wanted it to be the same for the API.
Switching to Serverless
I studied a very good ASP.NET Web API hosting tutorial with the Serverless framework and found that you only need to add a simple configuration file, one dependency and a small startup class to an existing API project. Deploy took about twenty seconds - much better than in Beanstalk. I think thanks to the built-in support for .NET Core in Lambda (however, only 2.2), while in Beanstalk it is only supported if you use independent docker. In any case, I was happy without thinking about auto-scaling groups, max instances and the like.
Performance testing
Serverless on AWS is Lambda, which really hosts your features, and the Front Gateway API, which allows you to add things like speed limits, API keys, and more. I hosted the Lambda features in the us-west-2 region, where the Beanstalk servers are. Then I configured the CloudFront instance to route requests from one of our games to the new serverless configuration, and the other to the old Beanstalk configuration. Then he launched a simple test on two URLs: one Serverless, the other Beanstalk. Both URLs invoked the same API, storing the same event in the database. I ran 100 queries for each, here are the results:
Performance comparison
In multiple runs, Serverless configuration was 15% slower (if anything, I ran it from Iceland, hence the big ping). The speed was a disappointment, but it remained quite high - I realized that there is some overhead for the API Gateway front in front of our API: there are too many functions, even if we do not use them. So, sad, but not critical!
Prices
Honestly, at first I did not think about prices. I just decided that paying for actual use would probably be cheaper than paying for instances 24/7. Therefore, it allowed the new serverless installation to work for several days, and then began to check accounts. Oops! The bill for Lambda + API Gateway has already exceeded one hundred dollars! At first I started messing with the Lambda setup, allocating less memory for it to save, but when I looked at what was going on well, it became obvious that the problem was with the gateway. Here are the API Gateway rates:
Gateway API Tariffs
Our API accepts about 10 million requests per day. This is approximately $ 35 per gateway only each day . In addition, Lambda costs about $ 10 per day, although this can be reduced by allocating less memory. Together it turns out about $ 45 per day, or $ 1350 per month , versus $ 164 per month on Elastic Beanstalk. Eight times more expensive ! I like new technologies and a quick deployment, but I'm not going to pay an extra $ 1200 per month for this. Back to Beanstalk!
Output
Probably, I should have familiarized myself with the prices in advance and made some mathematical calculations! But of course, then I would have to do real work, and not learn valuable technical skills! I am sure that in some situations, the Gateway and Lambda APIs are better than the Elastic Beanstalk. It’s just not our case. Perhaps if you use API keys, speed limits and other Gateway API functions, then it makes sense to pay $ 3.50 per million requests. We would be better off just putting a normal load balancer in front of Lambda. As far as I know, this is not possible, Gateway API is required for http access to Lambda.
But even if we just paid for Lambda, at $ 10 a day it would have turned out $ 300 a month instead of $ 164. We have many queries, but each query does very little: basically, one DB call per query. Perhaps heavier queries that use more computing time are better suited for Lambda, where you pay for 100 ms of computing time. Below is a report for one request. As you can see, we use 3.50 ms of computing time, but the invoice is set for 100 ms, this is not a weak rounding.
Lambda report for one request
Finally, I don’t criticize Gateway, Lambda or Serverless APIs at all. Just showing that for some workloads they are much more expensive than the boring old EC2 and Elastic Beanstalk. On what we will remain. It is also likely that there is a much better or more efficient way to configure the system, I am by no means an AWS expert, so if you see any flagrant errors, be sure to indicate in the comments.
All Articles