1
One Billion Row Challenge
curiouscoding.nlTable of Contents External links The problem Initial solution: 105s First flamegraph Bytes instead of strings: 72s Manual parsing: 61s Inline hash keys: 50s Faster hash function: 41s A new flame graph Perf it is Something simple: allocating the right size: 41s memchr for scanning: 47s memchr crate: 29s get_unchecked: 28s Manual SIMD: 29s Profiling Revisiting the key function: 23s PtrHash perfect hash function: 17s Larger masks: 15s Reduce pattern matching: 14s Memory map: 12s Parallelization: 2.0s Branchless parsing: 1.7s Purging all branches: 1.67s Some more attempts Faster perfect hashing: 1.55s Bug time: Back up to 1.71s Temperatures less than 100: 1.62s Computing min as a max: 1.50 Intermezzo: Hyperthreading: 1.34s Not parsing negative numbers: 1.48s More efficient parsing: 1.44s Fixing undefined behaviour: back to 1.56s Lazily subtracting b'0': 1.52s Min/max without parsing: 1.55s Parsing using a single multiplication: doesn’t work Parsing using a single multiplication does work after all! 1.48s A side note: ASCII Skip parsing using PDEP: 1.42s Improved A further note Branchy min/max: 1.37s No counting: 1.34s Arbitrary long city names: 1.34 4 entries in parallel: 1.23s Mmap per thread Reordering some operations: 1.19s Reordering more: 1.11s Even more ILP: 1.05 Compliance 1, OK I’ll count: 1.06 TODO Postscript A youtube video on this post is here.
You must log in or register to comment.