In my few past projects I used TypeScript framework - NestJS. And this framework is well integrated with TypeORM. Working with the database could be one of the bottlenecks because of slow queries, bad optimized DB structure, etc. But also we need to understand how fast work ORM which we use. My colleague finds an article which shows that TypeORM is not so great, you could check it here. So I decided to make my performance benchmark.  

I created a simple express.js project with 4 routes witch get data from the same DB, 2 for each ORM and one just for filling DB with mock data and start testing. The first difference which I faced was that the original bench was using .eager() method for getting relations that are deprecated now. So I changed it to recommended .withGraphFetched(). I implement 2 most common situations for getting information from DB - simple query (without joining relations) and with relations. I used PostgreSQL as DB. The results which I get are different from the results in the original article. 

Machine specs, where the tests were run:

Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   39 bits physical, 48 bits virtual
CPU(s):                          8
On-line CPU(s) list:             0-7
Thread(s) per core:              2
Core(s) per socket:              4
Socket(s):                       1
NUMA node(s):                    1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           158
Model name:                      Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
Stepping:                        9
CPU MHz:                         3689.928
CPU max MHz:                     3800,0000
CPU min MHz:                     800,0000

Objection.js simple query

Running 10s test @ http://localhost:3000/objection
10 connections

┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬──────────┐
│ Stat    │ 2.5% │ 50%  │ 97.5% │ 99%  │ Avg     │ Stdev   │ Max      │
├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼──────────┤
│ Latency │ 2 ms │ 3 ms │ 4 ms  │ 5 ms │ 2.85 ms │ 0.69 ms │ 11.58 ms │
└─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴──────────┘
┌───────────┬────────┬────────┬────────┬────────┬─────────┬─────────┬────────┐
│ Stat      │ 1%     │ 2.5%   │ 50%    │ 97.5%  │ Avg     │ Stdev   │ Min    │
├───────────┼────────┼────────┼────────┼────────┼─────────┼─────────┼────────┤
│ Req/Sec   │ 2457   │ 2457   │ 3123   │ 3217   │ 3042.82 │ 208.07  │ 2457   │
├───────────┼────────┼────────┼────────┼────────┼─────────┼─────────┼────────┤
│ Bytes/Sec │ 700 kB │ 700 kB │ 889 kB │ 916 kB │ 866 kB  │ 59.2 kB │ 700 kB │
└───────────┴────────┴────────┴────────┴────────┴─────────┴─────────┴────────┘

Req/Bytes counts sampled once per second.

33k requests in 11.04s, 9.53 MB read

Objection.js eager query 

Running 10s test @ http://localhost:3000/objection/eager
10 connections

┌─────────┬──────┬───────┬───────┬───────┬──────────┬────────┬──────────┐
│ Stat    │ 2.5% │ 50%   │ 97.5% │ 99%   │ Avg      │ Stdev  │ Max      │
├─────────┼──────┼───────┼───────┼───────┼──────────┼────────┼──────────┤
│ Latency │ 8 ms │ 10 ms │ 25 ms │ 29 ms │ 11.48 ms │ 4.7 ms │ 65.93 ms │
└─────────┴──────┴───────┴───────┴───────┴──────────┴────────┴──────────┘
┌───────────┬────────┬────────┬────────┬────────┬────────┬────────┬────────┐
│ Stat      │ 1%     │ 2.5%   │ 50%    │ 97.5%  │ Avg    │ Stdev  │ Min    │
├───────────┼────────┼────────┼────────┼────────┼────────┼────────┼────────┤
│ Req/Sec   │ 375    │ 375    │ 939    │ 1077   │ 834.7  │ 235.94 │ 375    │
├───────────┼────────┼────────┼────────┼────────┼────────┼────────┼────────┤
│ Bytes/Sec │ 164 kB │ 164 kB │ 412 kB │ 472 kB │ 366 kB │ 103 kB │ 164 kB │
└───────────┴────────┴────────┴────────┴────────┴────────┴────────┴────────┘

Req/Bytes counts sampled once per second.

8k requests in 10.04s, 3.66 MB read

TypeORM simple query

Running 10s test @ http://localhost:3000/typeorm
10 connections

┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬──────────┐
│ Stat    │ 2.5% │ 50%  │ 97.5% │ 99%  │ Avg     │ Stdev   │ Max      │
├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼──────────┤
│ Latency │ 1 ms │ 2 ms │ 3 ms  │ 3 ms │ 2.03 ms │ 0.34 ms │ 10.54 ms │
└─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴──────────┘
┌───────────┬────────┬────────┬─────────┬─────────┬─────────┬─────────┬────────┐
│ Stat      │ 1%     │ 2.5%   │ 50%     │ 97.5%   │ Avg     │ Stdev   │ Min    │
├───────────┼────────┼────────┼─────────┼─────────┼─────────┼─────────┼────────┤
│ Req/Sec   │ 3555   │ 3555   │ 4239    │ 4315    │ 4158    │ 212.23  │ 3554   │
├───────────┼────────┼────────┼─────────┼─────────┼─────────┼─────────┼────────┤
│ Bytes/Sec │ 956 kB │ 956 kB │ 1.14 MB │ 1.16 MB │ 1.12 MB │ 57.1 kB │ 956 kB │
└───────────┴────────┴────────┴─────────┴─────────┴─────────┴─────────┴────────┘

Req/Bytes counts sampled once per second.

46k requests in 11.03s, 12.3 MB read

TypeORM eager query

Running 10s test @ http://localhost:3000/typeorm/eager
10 connections

┌─────────┬──────┬──────┬───────┬──────┬─────────┬─────────┬──────────┐
│ Stat    │ 2.5% │ 50%  │ 97.5% │ 99%  │ Avg     │ Stdev   │ Max      │
├─────────┼──────┼──────┼───────┼──────┼─────────┼─────────┼──────────┤
│ Latency │ 3 ms │ 4 ms │ 5 ms  │ 5 ms │ 3.97 ms │ 0.69 ms │ 10.99 ms │
└─────────┴──────┴──────┴───────┴──────┴─────────┴─────────┴──────────┘
┌───────────┬────────┬────────┬────────┬────────┬────────┬─────────┬────────┐
│ Stat      │ 1%     │ 2.5%   │ 50%    │ 97.5%  │ Avg    │ Stdev   │ Min    │
├───────────┼────────┼────────┼────────┼────────┼────────┼─────────┼────────┤
│ Req/Sec   │ 2171   │ 2171   │ 2243   │ 2287   │ 2244.2 │ 29.81   │ 2171   │
├───────────┼────────┼────────┼────────┼────────┼────────┼─────────┼────────┤
│ Bytes/Sec │ 864 kB │ 864 kB │ 892 kB │ 910 kB │ 892 kB │ 11.8 kB │ 863 kB │
└───────────┴────────┴────────┴────────┴────────┴────────┴─────────┴────────┘

Req/Bytes counts sampled once per second.

22k requests in 10.03s, 8.92 MB read

 

Final results:

 TypeOrmObjection
simple query46k requests33k requests
eager query22k requests8k requests

 

 

Code for this test you could check on this repo, feel free to check it and add your comments bellow wink

Links

0 Comments

    Leave a comment