Introduction
I’ve started looking into Lambda a little more. As I code with 4 ‘main’ languages (C#, Go, Node, Python) I thought I’d see the relative performance of each as I’m quite happy to choose based on that simple criteria for some projects - the kind of stuff that doesn’t necessarily benefit too much from any language’s strong points and so could be done by all of them.
- The results are based on AWS overheads, and do not take into account throughput for actual running services.
- A language that appears slow here may process more requests under real usage; this does not test that aspect.
There is no concept in this head to head of actual performance, it is all about the memory and start-up overheads of individual invocations. I fully expect for example (though am not blinkered enough that I cannot be dissuaded when I test it) that the scripting languages (Node and especially Python) will perform better here than a JIT system like C#, and yet C# will probably perform a lot better when it is warmed up. We’ll see; this is just part one.
In order to avoid bias and to go with AWS best practices, I used the Lambda console to create the default functions then tested them from that console without any code changes. They are what Amazon consider to be ‘out of the box’ minimal functions.
I did 8 runs with cold starts at runs 1 and 7 and the details are below. This is not scientific as each run was a manual trigger in the console, but for these purposes that doesn’t matter.
Basic stats
Overall size, resource usage, and timings:
| Code Size | Resources | Max Time | |
|---|---|---|---|
| Go | 1.4 mb | 128 mb | 22 ms |
| Net 2.1 | 0.5 mb | 512 mb | 53 ms |
| Node | 0.3 mb | 128 mb | 44 ms |
| Python 3 | 0.3 mb | 128 mb | 21 ms |
Timings for the 8 runs, in milliseconds:
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | |
|---|---|---|---|---|---|---|---|---|
| Go | 0.7 | 0.4 | 5.8 | 36.9 | 0.4 | 23.9 | 15.6 | 16.8 |
| Net 2.1 | 540.9 | 0.3 | 0.2 | 0.4 | 0.2 | 0.2 | 517.6 | 0.3 |
| Node | 32.8 | 0.5 | 0.3 | 0.3 | 10.9 | 8.5 | 33.8 | 6.4 |
| Python 3 | 0.3 | 0.3 | 0.4 | 0.5 | 0.3 | 0.3 | 0.3 | 11.3 |
Notes
- As stated, there were cold starts on runs 1 and 7.
- I have no explanation for run 4 (Go), 5 (Node), 6 (Go), or 8 (various).
- Go and Python are better with resources, followed by Node, with C# trailing notably.
- C# has a relatively huge cold start time. Node is quite small, with Python and Go being negligible.
- Code size is not particularly relevant for any of the platforms (in terms of boilerplate overhead).
- Ignoring cold start, C# has the most consistent timings. Then Python, with Node and Go being a little more ‘bumpy’.
Stats-based ranking
| Resources | Cold Start | Timing | Consistency | |
|---|---|---|---|---|
| 1. | Python | Python | C# | C# |
| 2. | Go | Go | Python | Python |
| 3. | Node | Node | Node | Node |
| 4. | C# | C# | Go | Go |
Overall summary
In the end, they are all good enough. Depending upon the project I’ll choose the one that comes with the toolchain and library support I need. If I’m still not sure, then I will wait for my follow-up post which will deal with real performance under typical usage.