OK, 550000 pages rendered in 23 minutes isn’t bad. There is room for improvement of course, but in my perspective, comparing it the other static site generators, it s pretty darn good.
I’ve done some more structured testing (with default pagination). I created 1,000 random articles with simple tags/categories, and copied them into 20 different sections, for a total of 20,000 content files. Then I wrote a standalone script that would generate X taxonomies each containing Y terms, and insert 1-X random taxonomies into each article, each with 1-5 random terms.
Without the random taxonomies, build time was 268956 ms.
With one 1,000-term taxonomy, it was 290217 ms.
With ten 1,000-term taxonomies, it was 332273 ms.
With one 10,000-term taxonomy, it was 316232 ms.
With four 10,000-term taxonomies, it was 427258 ms.
With four 10,000-term taxonomies and 6-10 terms/taxonomy, it was still only 511538 ms:
0 draft content
0 future content
0 expired content
20000 regular pages created
80088 other pages created
20 non-page files copied
75566 paginator pages created
10000 drove created
9999 pneumonographic created
10 categories created
10 tags created
9999 pleasingness created
9999 smidgen created
total in 511538 ms
So I decided to go for broke, and generated 20 10,000-term taxonomies with 1-5 terms/taxonomy. The build has been running for 50 minutes so far, using only a single core and 4.2 GB of RAM, not spinning up the fans, and has only written out the static
files.
If I’m reading the stack trace correctly, only one thread is active, and it’s spending all of its time in assemble().
Update: Final results after 68 minutes:
0 draft content
0 future content
0 expired content
20000 regular pages created
371332 other pages created
0 non-page files copied
207794 paginator pages created
9253 formalesque created
9211 ankyloglossia created
9285 accidie created
9294 cholestanol created
9291 hala created
9280 undisgraced created
9273 brocho created
9270 subsist created
9252 featherless created
9275 turner created
9290 unawfully created
9280 overwalk created
9300 dicker created
9246 electoral created
9302 antalkali created
9296 overdaintily created
9284 tomeful created
9316 extrafloral created
9322 coruscation created
10 categories created
9283 scranny created
10 tags created
total in 4073392 ms
So, 5x the number of taxonomies/terms, 10x the runtime, and most of that was spent in a single-threaded routine that was neither reading from nor writing to the disk.
-j
Sure: taxonomies.pl.
Usage is simple: feed it a bunch of filenames on STDIN, and it will add random taxonomies to their TOML front matter. So, to create 3 taxonomies with 1000 terms each, and then add 1-3 of them with 1-5 randomly-selected terms to each article:
find content -name '*.md' | taxonomies.pl -T 3 -t 1000 -m 5
The thousand content files that I generated with my wikiblog.sh script are here (5MB tarball). I just copied them repeatedly into different sections to increase the article count, and then ran the taxonomy-adder on the results.
-j
Holy shnikes. Hugo is blisteringly fast.
@jgreely I’d be curious to see something more real-world-ish; e.g, a 10k pages, 10 sections, 5 taxonomies, with maybe 50 terms each (this would be a formidable group of metadata to manage). Also, what’s the templating like?
I seem to recall @budparr was saying he was working on a decent-sized site with some complex templating. Maybe he can add some insight into an example in the wild.
Done.
Without random taxonomies:
0 draft content
0 future content
0 expired content
10000 regular pages created
66 other pages created
0 non-page files copied
11128 paginator pages created
10 categories created
10 tags created
total in 81768 ms
Adding 5 taxonomies of 50 terms, 1-5 tax/article with 1-5 terms/tax:
0 draft content
0 future content
0 expired content
10000 regular pages created
576 other pages created
0 non-page files copied
18669 paginator pages created
50 psychological created
50 loudish created
50 bullbaiting created
10 categories created
10 tags created
50 pseudomodest created
50 unerrable created
total in 91408 ms
config.toml:
languageCode = "en-us"
title = "5 random taxonomies"
baseURL = "https://example.com/"
theme = "mt2-theme"
[taxonomies]
category = "categories"
tag = "tags"
pseudomodest = "pseudomodest"
unerrable = "unerrable"
psychological = "psychological"
loudish = "loudish"
bullbaiting = "bullbaiting"
I used the (unpublished as yet) theme for my blog, because it paginates sections and taxonomies. If you have a specific theme you think would work well for testing, I can try it. It’s a bit painful to wade through the gallery looking for features like pagination (for instance, I tried hugo-octopress, but all it generated for taxonomies was RSS feeds, so it only created 100 paginator pages, and finished in 20 seconds).
-j
for the record on Amazon r3.2xlarge 64go de RAM 8cpu
Built site for language en:
0 draft content
0 future content
0 expired content
220341 regular pages created
24 other pages created
0 non-page files copied
10 paginator pages created
6 tags created
0 categories created
total in 209277 ms
and this is not lorem ipsum pages but real pages (with real content), some pages are build with a FML 300ko json
@jonathanulco that is really cool, and it would be really interesting if you could elaborate a little about what kind of project this is.
I work for a little startup who create a proximity social network, these are the external pages created by users of the service.
P.S : For improve my service, i’m waiting for the incremental build on hugo
I received the bill from Amazon : $0.67
What’s the bill for?
1 Hugo build with 200K+ pages, see above.
@JLKM @jgreely did you by any change use TOML as page front matter?
If so, see https://github.com/spf13/hugo/issues/3464
And
https://github.com/spf13/hugo/issues/3541
If yes, it would also be interesting if you could repeat the test with YAML.
Yes, for all of my tests. I don’t have time to set up the exact same tests again at the moment, but I do have a work-in-progress site with 50,000+ recipes in 67 sections and 782 categories (0-8 categories per article, with most having 1-2). Using 0.21 on a Mac, here’s the TOML versus YAML comparison.
TOML:
0 draft content
0 future content
0 expired content
56842 regular pages created
851 other pages created
0 non-page files copied
9198 paginator pages created
782 categories created
total in 572439 ms
YAML:
Built site for language en:
0 draft content
0 future content
0 expired content
56842 regular pages created
851 other pages created
0 non-page files copied
9198 paginator pages created
782 categories created
total in 560110 ms
Soon as I have a chance, I’ll replicate the torture test, since this one doesn’t show a huge difference. Oh, and before I forget to mention it, in all of these tests, I’ve been working in a directory that’s excluded from Spotlight indexing, which can seriously interfere with timing.
Amusing side note: when I started building the recipe site (which bulk-converts MasterCook MX2 files from various archives into Hugo content files), I grabbed the Zen theme for a simple, clean look, and watched the first build eat my memory and disk, because the theme embeds links to every content page in the navbar and sidebar. Every file in public
was over 5 MB in size, and top
reported it using 40GB of compressed pages when I killed it.
-j
@jgreely thanks, looking at your “monster test”, you have lots of different taxonomy terms (not very realistic, maybe?), which I have not tested well – I will add that variant to my benchmarks as well.
I’d have called it completely unrealistic if it weren’t for JKLM’s original site, which has 22 taxonomies ranging from 6 to 8,163 terms (mean 3,763, median 345).
-j
I replicated the big test and kicked it off just before going to bed last night. I took the same 1,000 randomly-generated articles, replicated them into a total of 20 sections, and then used my script to add 20 10,000-term taxonomies. I ended up with a total of four sites, generating the YAML versions with rsync
and hugo convert --unsafe toYAML
:
- TOML, no additional taxonomies: 363945 ms
- YAML, no additional taxonomies: 325886 ms
- TOML, 20x big taxonomies: 3460447 ms
- YAML, 20x big taxonomies: 3548300 ms
Yes, the baseline test took 11% longer with TOML, while the really big test took 2.5% longer with YAML. I suspect that any parsing issues are small compared to the amount of time it spends in the single-threaded assemble() function, which is what the stack traces of my earlier big test showed.
Ideally, I’d run each test 10 times to make sure the timing differences are real, but at the very least, I can say that YAML isn’t obviously faster on a big site with lots of taxonomies.
-j
Yes, I have reproduced it in my benchmarks now – the TOML issue does have a fair effect on “normal sites”, but not what you’re dealing with.
I have looked into this, and for the “multiple tags per page” case, my surprising conclusion is that the front matter handling of lists is the big factor here.
I did a quick hack where I accepted tags as CSV string and just split it into tags:
benchmark old ns/op new ns/op delta
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=100,tags_per_page=20-4 4008327269 68389054 -98.29%
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=100,tags_per_page=50-4 9946870164 80731086 -99.19%
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=100,tags_per_page=80-4 15994140132 87300135 -99.45%
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=500,tags_per_page=20-4 4004805422 77224226 -98.07%
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=500,tags_per_page=50-4 9886487339 87193850 -99.12%
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=500,tags_per_page=80-4 15525606788 101154964 -99.35%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=100,tags_per_page=20-4 73795543 69456250 -5.88%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=100,tags_per_page=50-4 94281466 76522987 -18.84%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=100,tags_per_page=80-4 113422149 87748893 -22.64%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=500,tags_per_page=20-4 84782504 75945331 -10.42%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=500,tags_per_page=50-4 104024340 90037331 -13.45%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=500,tags_per_page=80-4 126581039 109897153 -13.18%
benchmark old allocs new allocs delta
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=100,tags_per_page=20-4 32880961 325913 -99.01%
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=100,tags_per_page=50-4 81733175 356537 -99.56%
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=100,tags_per_page=80-4 130582482 387931 -99.70%
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=500,tags_per_page=20-4 32902873 338776 -98.97%
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=500,tags_per_page=50-4 81752074 369615 -99.55%
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=500,tags_per_page=80-4 130602150 400849 -99.69%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=100,tags_per_page=20-4 439205 328285 -25.25%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=100,tags_per_page=50-4 636468 359278 -43.55%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=100,tags_per_page=80-4 832516 390919 -53.04%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=500,tags_per_page=20-4 461346 341230 -26.04%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=500,tags_per_page=50-4 668633 372878 -44.23%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=500,tags_per_page=80-4 875490 404527 -53.79%
benchmark old bytes new bytes delta
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=100,tags_per_page=20-4 1279495480 37490478 -97.07%
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=100,tags_per_page=50-4 3165165576 40039393 -98.73%
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=100,tags_per_page=80-4 5123007552 42866740 -99.16%
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=500,tags_per_page=20-4 1250653472 38725958 -96.90%
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=500,tags_per_page=50-4 3131357648 41197570 -98.68%
BenchmarkSiteBuilding/YAML,num_pages=500,num_tags=500,tags_per_page=80-4 5058255136 44584950 -99.12%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=100,tags_per_page=20-4 38848232 36935079 -4.92%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=100,tags_per_page=50-4 45143742 40513162 -10.26%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=100,tags_per_page=80-4 51166390 45180727 -11.70%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=500,tags_per_page=20-4 41045064 38201983 -6.93%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=500,tags_per_page=50-4 48266240 41843627 -13.31%
BenchmarkSiteBuilding/TOML,num_pages=500,num_tags=500,tags_per_page=80-4 56945800 47070758 -17.34% 2d
Note that in the above, the TOML library used is BurntSushi (which is in the current Hugo master), which is much faster than in the Hugo release version.
The work done in the above is the site building excluding rendering.
This is a comparision of the tags front matter handling of the different file types (this is not BurntSushi):
BenchmarkFrontmatterTags/JSON:1-4 1000000 2039.00 ns/op 912 B/op 20 allocs/op
BenchmarkFrontmatterTags/JSON:11-4 300000 5202.00 ns/op 1640 B/op 44 allocs/op
BenchmarkFrontmatterTags/JSON:21-4 200000 7993.00 ns/op 2392 B/op 65 allocs/op
BenchmarkFrontmatterTags/YAML:1-4 200000 9359.00 ns/op 5928 B/op 66 allocs/op
BenchmarkFrontmatterTags/YAML:11-4 100000 21218.00 ns/op 8408 B/op 140 allocs/op
BenchmarkFrontmatterTags/YAML:21-4 50000 32852.00 ns/op 10920 B/op 211 allocs/op
BenchmarkFrontmatterTags/TOML:1-4 100000 21505.00 ns/op 9231 B/op 173 allocs/op
BenchmarkFrontmatterTags/TOML:11-4 20000 82919.00 ns/op 19808 B/op 625 allocs/op
BenchmarkFrontmatterTags/TOML:21-4 10000 141847.00 ns/op 31200 B/op 1106 allocs/op
```
This is the same test with BurntSushi TOML lib:
```
benchmark iter time/iter bytes alloc allocs
--------- ---- --------- ----------- ------
BenchmarkFrontmatterTags/JSON:1-4 500000 2161.00 ns/op 912 B/op 20 allocs/op
BenchmarkFrontmatterTags/JSON:11-4 200000 5565.00 ns/op 1640 B/op 44 allocs/op
BenchmarkFrontmatterTags/JSON:21-4 200000 7907.00 ns/op 2392 B/op 65 allocs/op
BenchmarkFrontmatterTags/YAML:1-4 200000 9511.00 ns/op 5928 B/op 66 allocs/op
BenchmarkFrontmatterTags/YAML:11-4 100000 21651.00 ns/op 8408 B/op 140 allocs/op
BenchmarkFrontmatterTags/YAML:21-4 50000 33415.00 ns/op 10920 B/op 211 allocs/op
BenchmarkFrontmatterTags/TOML:1-4 200000 7670.00 ns/op 2912 B/op 60 allocs/op
BenchmarkFrontmatterTags/TOML:11-4 100000 17948.00 ns/op 5184 B/op 138 allocs/op
BenchmarkFrontmatterTags/TOML:21-4 50000 28741.00 ns/op 7536 B/op 210 allocs/op
```
The tag count is 1, 11 and 21.