Lemmy: Bestiverse
  • Communities
  • Create Post
  • Create Community
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
RSS BotMB to Lobste.rsEnglish · 3 days ago

One Billion Row Challenge (2024)

curiouscoding.nl

external-link
message-square
0
fedilink
1
external-link

One Billion Row Challenge (2024)

curiouscoding.nl

RSS BotMB to Lobste.rsEnglish · 3 days ago
message-square
0
fedilink
One Billion Row Challenge
curiouscoding.nl
external-link
Table of Contents External links The problem Initial solution: 105s First flamegraph Bytes instead of strings: 72s Manual parsing: 61s Inline hash keys: 50s Faster hash function: 41s A new flame graph Perf it is Something simple: allocating the right size: 41s memchr for scanning: 47s memchr crate: 29s get_unchecked: 28s Manual SIMD: 29s Profiling Revisiting the key function: 23s PtrHash perfect hash function: 17s Larger masks: 15s Reduce pattern matching: 14s Memory map: 12s Parallelization: 2.0s Branchless parsing: 1.7s Purging all branches: 1.67s Some more attempts Faster perfect hashing: 1.55s Bug time: Back up to 1.71s Temperatures less than 100: 1.62s Computing min as a max: 1.50 Intermezzo: Hyperthreading: 1.34s Not parsing negative numbers: 1.48s More efficient parsing: 1.44s Fixing undefined behaviour: back to 1.56s Lazily subtracting b'0': 1.52s Min/max without parsing: 1.55s Parsing using a single multiplication: doesn’t work Parsing using a single multiplication does work after all! 1.48s A side note: ASCII Skip parsing using PDEP: 1.42s Improved A further note Branchy min/max: 1.37s No counting: 1.34s Arbitrary long city names: 1.34 4 entries in parallel: 1.23s Mmap per thread Reordering some operations: 1.19s Reordering more: 1.11s Even more ILP: 1.05 Compliance 1, OK I’ll count: 1.06 TODO Postscript A youtube video on this post is here.

Comments

alert-triangle
You must log in or register to comment.

Lobste.rs

lobsters

Subscribe from Remote Instance

You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !lobsters@lemmy.bestiver.se
lock
Community locked: only moderators can create posts. You can still comment on posts.

RSS Feed of lobste.rs

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 22 users / day
  • 128 users / week
  • 334 users / month
  • 1.21K users / 6 months
  • 2 local subscribers
  • 231 subscribers
  • 6.93K Posts
  • 324 Comments
  • Modlog
  • mods:
  • patrick
  • RSS Bot
  • BE: 0.19.5
  • Modlog
  • Instances
  • Docs
  • Code
  • join-lemmy.org