Yes. That info is on the repo, but I can tell you here. 42 workers in the server, and 400 workers in the client. Let me know if you have additional questions. With local storage, nvme, we reached 2.08m.
Hi, Ignacio here, Lead of Basekick Labs, Creators of Arc. Thanks for the feedback! You were right, we investigated and found optimization opportunities.
What we did:
• MessagePack streaming decoder (msgpack.Unpacker vs unpackb)
• Columnar Polars DataFrame construction
Results:
• Throughput: 1.95M → 2.01M RPS (+3.1%)
• Latency p95: 184ms → 147ms (-20.3%)
• Latency p99: 395ms → 317ms (-19.6%)
• Memory usage: -10-20%
The MessagePack endpoint already used Direct Arrow (zero-copy), but your comment helped us optimize both write paths.
CPU and Ram is also there, it’s a M3 Pro max. 14 cores, 36gb of RAM. CPU during those test at 85%.
Yes. That info is on the repo, but I can tell you here. 42 workers in the server, and 400 workers in the client. Let me know if you have additional questions. With local storage, nvme, we reached 2.08m.
Hi, Ignacio here, Lead of Basekick Labs, Creators of Arc. Thanks for the feedback! You were right, we investigated and found optimization opportunities.What we did:• MessagePack streaming decoder (msgpack.Unpacker vs unpackb)• Columnar Polars DataFrame constructionResults:• Throughput: 1.95M → 2.01M RPS (+3.1%)• Latency p95: 184ms → 147ms (-20.3%)• Latency p99: 395ms → 317ms (-19.6%)• Memory usage: -10-20%The MessagePack endpoint already used Direct Arrow (zero-copy), but your comment helped us optimize both write paths.Details:https://github.com/basekick-labs/arc/blob/main/docs/WRITE\_OPTIMIZATIONS.mdOpen source works! 🚀