Runtime: garbage collector found invalid heap pointer

support
1

I was moving right along with hugo until I tried to generate a couple thousand .md content files, corresponding .json files in data and corresponding images in static. Here’s what I received after trying to run hugo server thereafter. I couldn’t copy it all due to size limits in this forum, but I included a healthy excerpt.

Any ideas? Am I throwing too much at it?

PS C:\dev\hugo\micro.moda> hugo server –
runtime: garbage collector found invalid heap pointer *(0xb705a0+0x2e0)=0x52e68428 s=nil
fatal error: invalid heap pointer

runtime stack:
runtime.throw(0xb5e403)
/usr/local/Cellar/go/1.4/libexec/src/runtime/panic.go:491 +0x83 fp=0xcfc58 sp=0xcfc40
scanblock(0xb705a0, 0x9040, 0x18c708)
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:378 +0x48d fp=0xcfcf8 sp=0xcfc58
markroot(0x1292c0a0, 0x1)
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:496 +0x139 fp=0xcfd30 sp=0xcfcf8
runtime.parfordo(0x1292c0a0)
/usr/local/Cellar/go/1.4/libexec/src/runtime/parfor.c:91 +0x115 fp=0xcfd8c sp=0xcfd30
gc(0xcfec8)
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:1439 +0x1fb fp=0xcfeb8 sp=0xcfd8c
runtime.gc_m()
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:1368 +0xd2 fp=0xcfed8 sp=0xcfeb8
runtime.onM(0xb71878)
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:266 +0x50 fp=0xcfedc sp=0xcfed8
runtime.mstart()
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.c:818 fp=0xcfee0 sp=0xcfedc

goroutine 1 [garbage collection]:
runtime.switchtoM()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:208 fp=0x12977004 sp=0x12977000
runtime.gogc(0x0)
/usr/local/Cellar/go/1.4/libexec/src/runtime/malloc.go:469 +0x1aa fp=0x12977024 sp=0x12977004
runtime.mallocgc(0x90, 0x877980, 0x0, 0x65734401)
/usr/local/Cellar/go/1.4/libexec/src/runtime/malloc.go:341 +0x2c4 fp=0x1297707c sp=0x12977024
runtime.newarray(0x877980, 0x1, 0x7bc7387e)
/usr/local/Cellar/go/1.4/libexec/src/runtime/malloc.go:365 +0xb5 fp=0x1297709c sp=0x1297707c
runtime.mapassign1(0x7debc0, 0x657385c0, 0x1297712c, 0x12977124)
/usr/local/Cellar/go/1.4/libexec/src/runtime/hashmap.go:395 +0xbc fp=0x129770f8 sp=0x1297709c
encoding/json.(*decodeState).objectInterface(0x65645280, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1d6 fp=0x12977144 sp=0x129770f8
encoding/json.(*decodeState).valueInterface(0x65645280, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x1297716c sp=0x12977144
encoding/json.(*decodeState).objectInterface(0x65645280, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x129771b8 sp=0x1297716c
encoding/json.(*decodeState).valueInterface(0x65645280, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x129771e0 sp=0x129771b8
encoding/json.(*decodeState).objectInterface(0x65645280, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x1297722c sp=0x129771e0
encoding/json.(*decodeState).valueInterface(0x65645280, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x12977254 sp=0x1297722c
encoding/json.(*decodeState).objectInterface(0x65645280, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x129772a0 sp=0x12977254
encoding/json.(*decodeState).valueInterface(0x65645280, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x129772c8 sp=0x129772a0
encoding/json.(*decodeState).objectInterface(0x65645280, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x12977314 sp=0x129772c8
encoding/json.(*decodeState).valueInterface(0x65645280, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x1297733c sp=0x12977314
encoding/json.(*decodeState).objectInterface(0x65645280, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x12977388 sp=0x1297733c
encoding/json.(*decodeState).valueInterface(0x65645280, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x129773b0 sp=0x12977388
encoding/json.(*decodeState).objectInterface(0x65645280, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x129773fc sp=0x129773b0
encoding/json.(*decodeState).valueInterface(0x65645280, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x12977424 sp=0x129773fc
encoding/json.(*decodeState).objectInterface(0x65645280, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x12977470 sp=0x12977424
encoding/json.(*decodeState).valueInterface(0x65645280, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x12977498 sp=0x12977470
encoding/json.(*decodeState).objectInterface(0x65645280, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x129774e4 sp=0x12977498
encoding/json.(*decodeState).valueInterface(0x65645280, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x1297750c sp=0x129774e4
encoding/json.(*decodeState).objectInterface(0x65645280, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x12977558 sp=0x1297750c
encoding/json.(*decodeState).valueInterface(0x65645280, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x12977580 sp=0x12977558
encoding/json.(*decodeState).objectInterface(0x65645280, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x129775cc sp=0x12977580
encoding/json.(*decodeState).valueInterface(0x65645280, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x129775f4 sp=0x129775cc
encoding/json.(*decodeState).objectInterface(0x65645280, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x12977640 sp=0x129775f4
encoding/json.(*decodeState).valueInterface(0x65645280, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x12977668 sp=0x12977640
encoding/json.(*decodeState).objectInterface(0x65645280, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x129776b4 sp=0x12977668
encoding/json.(*decodeState).valueInterface(0x65645280, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x129776dc sp=0x129776b4
encoding/json.(*decodeState).objectInterface(0x65645280, 0x6563e970)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x12977728 sp=0x129776dc
encoding/json.(*decodeState).object(0x65645280, 0x7dbd80, 0x6563e970, 0xd4)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:497 +0x306 fp=0x129778c8 sp=0x12977728
encoding/json.(*decodeState).value(0x65645280, 0x7c5d80, 0x6563e970, 0x16)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:288 +0x2cd fp=0x12977928 sp=0x129778c8
encoding/json.(*decodeState).unmarshal(0x65645280, 0x7c5d80, 0x6563e970, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:149 +0x177 fp=0x12977978 sp=0x12977928
encoding/json.Unmarshal(0x6564e000, 0x39e53, 0x39e53, 0x7c5d80, 0x6563e970, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:78 +0xf7 fp=0x129779b0 sp=0x12977978
github.com/spf13/hugo/parser.HandleJsonMetaData(0x6564e000, 0x39e53, 0x39e53, 0x0, 0x0, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/parser/frontmatter.go:189 +0x81 fp=0x129779dc sp=0x129779b0
github.com/spf13/hugo/hugolib.readData(0x129a0900, 0x0, 0x0, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/hugolib/site.go:321 +0xf6 fp=0x12977a3c sp=0x129779dc
github.com/spf13/hugo/hugolib.(*Site).loadData(0x129f0d00, 0x12977bfc, 0x1, 0x2, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/hugolib/site.go:287 +0x35c fp=0x12977b9c sp=0x12977a3c
github.com/spf13/hugo/hugolib.(*Site).Process(0x129f0d00, 0x18e040, 0x1293fc40)
/Users/spf13/gopath/src/github.com/spf13/hugo/hugolib/site.go:347 +0x33f fp=0x12977c10 sp=0x12977b9c
github.com/spf13/hugo/hugolib.(*Site).Build(0x129f0d00, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/hugolib/site.go:241 +0x38 fp=0x12977c7c sp=0x12977c10
github.com/spf13/hugo/commands.buildSite(0x12977d13, 0x1, 0x1, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/commands/hugo.go:342 +0xb3 fp=0x12977cf4 sp=0x12977c7c
github.com/spf13/hugo/commands.build(0x12977dae, 0x1, 0x1)
/Users/spf13/gopath/src/github.com/spf13/hugo/commands/hugo.go:254 +0x216 fp=0x12977d8c sp=0x12977cf4
github.com/spf13/hugo/commands.server(0xb6dd40, 0x1293e528, 0x0, 0x1)
/Users/spf13/gopath/src/github.com/spf13/hugo/commands/server.go:97 +0x43d fp=0x12977e58 sp=0x12977d8c
github.com/spf13/cobra.(*Command).execute(0xb6dd40, 0x1293e510, 0x1, 0x1, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/cobra/command.go:399 +0x2a1 fp=0x12977eb4 sp=0x12977e58
github.com/spf13/cobra.(*Command).findAndExecute(0xb6d840, 0x12927768, 0x2, 0x2, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/cobra/command.go:366 +0xa1 fp=0x12977ef4 sp=0x12977eb4
github.com/spf13/cobra.(*Command).Execute(0xb6d840, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/cobra/command.go:452 +0x83a fp=0x12977fa0 sp=0x12977ef4
github.com/spf13/hugo/commands.Execute()
/Users/spf13/gopath/src/github.com/spf13/hugo/commands/hugo.go:64 +0x2f fp=0x12977fc0 sp=0x12977fa0
main.main()
/Users/spf13/gopath/src/github.com/spf13/hugo/main.go:24 +0x2f fp=0x12977fcc sp=0x12977fc0
runtime.main()
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:63 +0xcc fp=0x12977ff0 sp=0x12977fcc
runtime.goexit()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:2287 +0x1 fp=0x12977ff4 sp=0x12977ff0

goroutine 2 [force gc (idle)]:
runtime.gopark(0x42c540, 0xb71358, 0x907de8, 0xf)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:130 +0xde fp=0x1293afcc sp=0x1293afb4
runtime.goparkunlock(0xb71358, 0x907de8, 0xf)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:136 +0x42 fp=0x1293afe0 sp=0x1293afcc
runtime.forcegchelper()
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:99 +0xa3 fp=0x1293aff0 sp=0x1293afe0
runtime.goexit()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:2287 +0x1 fp=0x1293aff4 sp=0x1293aff0
created by runtime.init┬╖4
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:87 +0x25

goroutine 3 [GC sweep wait]:
runtime.gopark(0x42c540, 0xb79e64, 0x9006c8, 0xd)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:130 +0xde fp=0x1293cfcc sp=0x1293cfb4
runtime.goparkunlock(0xb79e64, 0x9006c8, 0xd)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:136 +0x42 fp=0x1293cfe0 sp=0x1293cfcc
runtime.bgsweep()
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.go:98 +0x97 fp=0x1293cff0 sp=0x1293cfe0
runtime.goexit()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:2287 +0x1 fp=0x1293cff4 sp=0x1293cff0
created by gc
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:1383

goroutine 4 [finalizer wait]:
runtime.gopark(0x42c540, 0xb79e60, 0x9079a8, 0xe)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:130 +0xde fp=0x12937f98 sp=0x12937f80
runtime.goparkunlock(0xb79e60, 0x9079a8, 0xe)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:136 +0x42 fp=0x12937fac sp=0x12937f98
runtime.runfinq()
/usr/local/Cellar/go/1.4/libexec/src/runtime/malloc.go:727 +0x9c fp=0x12937ff0 sp=0x12937fac
runtime.goexit()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:2287 +0x1 fp=0x12937ff4 sp=0x12937ff0
created by runtime.createfing
/usr/local/Cellar/go/1.4/libexec/src/runtime/malloc.go:707 +0x59
PS C:\dev\hugo\micro.moda> hugo server --watch | Out-File c:\dev\hugo\startup.txt
runtime: garbage collector found invalid heap pointer *(0xb705a0+0x12ac)=0x42cb0000 s=nil
fatal error: invalid heap pointer

runtime stack:
runtime.throw(0xb5e403)
/usr/local/Cellar/go/1.4/libexec/src/runtime/panic.go:491 +0x83 fp=0xcfc58 sp=0xcfc40
scanblock(0xb705a0, 0x9040, 0x1bc708)
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:378 +0x48d fp=0xcfcf8 sp=0xcfc58
markroot(0x1274c0a0, 0x1)
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:496 +0x139 fp=0xcfd30 sp=0xcfcf8
runtime.parfordo(0x1274c0a0)
/usr/local/Cellar/go/1.4/libexec/src/runtime/parfor.c:91 +0x115 fp=0xcfd8c sp=0xcfd30
gc(0xcfec8)
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:1439 +0x1fb fp=0xcfeb8 sp=0xcfd8c
runtime.gc_m()
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:1368 +0xd2 fp=0xcfed8 sp=0xcfeb8
runtime.onM(0xb71878)
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:266 +0x50 fp=0xcfedc sp=0xcfed8
runtime.mstart()
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.c:818 fp=0xcfee0 sp=0xcfedc

goroutine 1 [garbage collection]:
runtime.switchtoM()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:208 fp=0x127970d8 sp=0x127970d4
runtime.gogc(0x0)
/usr/local/Cellar/go/1.4/libexec/src/runtime/malloc.go:469 +0x1aa fp=0x127970f8 sp=0x127970d8
runtime.mallocgc(0x30, 0x0, 0x3, 0x30)
/usr/local/Cellar/go/1.4/libexec/src/runtime/malloc.go:341 +0x2c4 fp=0x12797150 sp=0x127970f8
runtime.rawstring(0x23, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/runtime/string.go:195 +0x88 fp=0x12797168 sp=0x12797150
runtime.slicebytetostring(0x44649fb0, 0x23, 0x30, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/runtime/string.go:69 +0x38 fp=0x127971b0 sp=0x12797168
encoding/json.unquote(0x44649fb0, 0x23, 0x30, 0x0, 0x0, 0x44652301)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:968 +0x79 fp=0x127971d0 sp=0x127971b0
encoding/json.(*decodeState).literalInterface(0x442d1380, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:933 +0xee fp=0x1279722c sp=0x127971d0
encoding/json.(*decodeState).valueInterface(0x442d1380, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:836 +0x55 fp=0x12797254 sp=0x1279722c
encoding/json.(*decodeState).objectInterface(0x442d1380, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x127972a0 sp=0x12797254
encoding/json.(*decodeState).valueInterface(0x442d1380, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x127972c8 sp=0x127972a0
encoding/json.(*decodeState).objectInterface(0x442d1380, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x12797314 sp=0x127972c8
encoding/json.(*decodeState).valueInterface(0x442d1380, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x1279733c sp=0x12797314
encoding/json.(*decodeState).objectInterface(0x442d1380, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x12797388 sp=0x1279733c
encoding/json.(*decodeState).valueInterface(0x442d1380, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x127973b0 sp=0x12797388
encoding/json.(*decodeState).objectInterface(0x442d1380, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x127973fc sp=0x127973b0
encoding/json.(*decodeState).valueInterface(0x442d1380, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x12797424 sp=0x127973fc
encoding/json.(*decodeState).objectInterface(0x442d1380, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x12797470 sp=0x12797424
encoding/json.(*decodeState).valueInterface(0x442d1380, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x12797498 sp=0x12797470
encoding/json.(*decodeState).objectInterface(0x442d1380, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x127974e4 sp=0x12797498
encoding/json.(*decodeState).valueInterface(0x442d1380, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x1279750c sp=0x127974e4
encoding/json.(*decodeState).objectInterface(0x442d1380, 0x9)
– truncated here –
github.com/spf13/hugo/parser.HandleJsonMetaData(0x44422000, 0xc2bc5, 0xc2bc5, 0x0, 0x0, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/parser/frontmatter.go:189 +0x81 fp=0x127979dc sp=0x127979b0
github.com/spf13/hugo/hugolib.readData(0x1280f580, 0x0, 0x0, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/hugolib/site.go:321 +0xf6 fp=0x12797a3c sp=0x127979dc
github.com/spf13/hugo/hugolib.(*Site).loadData(0x127f7500, 0x12797bfc, 0x1, 0x2, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/hugolib/site.go:287 +0x35c fp=0x12797b9c sp=0x12797a3c
github.com/spf13/hugo/hugolib.(*Site).Process(0x127f7500, 0x1be040, 0x1275fe78)
/Users/spf13/gopath/src/github.com/spf13/hugo/hugolib/site.go:347 +0x33f fp=0x12797c10 sp=0x12797b9c
github.com/spf13/hugo/hugolib.(*Site).Build(0x127f7500, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/hugolib/site.go:241 +0x38 fp=0x12797c7c sp=0x12797c10
github.com/spf13/hugo/commands.buildSite(0x12797d13, 0x1, 0x1, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/commands/hugo.go:342 +0xb3 fp=0x12797cf4 sp=0x12797c7c
github.com/spf13/hugo/commands.build(0x12797dae, 0x1, 0x1)
/Users/spf13/gopath/src/github.com/spf13/hugo/commands/hugo.go:254 +0x216 fp=0x12797d8c sp=0x12797cf4
github.com/spf13/hugo/commands.server(0xb6dd40, 0x1275e528, 0x0, 0x1)
/Users/spf13/gopath/src/github.com/spf13/hugo/commands/server.go:97 +0x43d fp=0x12797e58 sp=0x12797d8c
github.com/spf13/cobra.(*Command).execute(0xb6dd40, 0x1275e510, 0x1, 0x1, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/cobra/command.go:399 +0x2a1 fp=0x12797eb4 sp=0x12797e58
github.com/spf13/cobra.(*Command).findAndExecute(0xb6d840, 0x12747768, 0x2, 0x2, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/cobra/command.go:366 +0xa1 fp=0x12797ef4 sp=0x12797eb4
github.com/spf13/cobra.(*Command).Execute(0xb6d840, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/cobra/command.go:452 +0x83a fp=0x12797fa0 sp=0x12797ef4
github.com/spf13/hugo/commands.Execute()
/Users/spf13/gopath/src/github.com/spf13/hugo/commands/hugo.go:64 +0x2f fp=0x12797fc0 sp=0x12797fa0
main.main()
/Users/spf13/gopath/src/github.com/spf13/hugo/main.go:24 +0x2f fp=0x12797fcc sp=0x12797fc0
runtime.main()
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:63 +0xcc fp=0x12797ff0 sp=0x12797fcc
runtime.goexit()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:2287 +0x1 fp=0x12797ff4 sp=0x12797ff0

goroutine 2 [force gc (idle)]:
runtime.gopark(0x42c540, 0xb71358, 0x907de8, 0xf)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:130 +0xde fp=0x1275afcc sp=0x1275afb4
runtime.goparkunlock(0xb71358, 0x907de8, 0xf)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:136 +0x42 fp=0x1275afe0 sp=0x1275afcc
runtime.forcegchelper()
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:99 +0xa3 fp=0x1275aff0 sp=0x1275afe0
runtime.goexit()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:2287 +0x1 fp=0x1275aff4 sp=0x1275aff0
created by runtime.init┬╖4
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:87 +0x25

goroutine 3 [GC sweep wait]:
runtime.gopark(0x42c540, 0xb79e64, 0x9006c8, 0xd)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:130 +0xde fp=0x1275cfcc sp=0x1275cfb4
runtime.goparkunlock(0xb79e64, 0x9006c8, 0xd)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:136 +0x42 fp=0x1275cfe0 sp=0x1275cfcc
runtime.bgsweep()
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.go:98 +0x97 fp=0x1275cff0 sp=0x1275cfe0
runtime.goexit()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:2287 +0x1 fp=0x1275cff4 sp=0x1275cff0
created by gc
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:1383

goroutine 4 [finalizer wait]:
runtime.gopark(0x42c540, 0xb79e60, 0x9079a8, 0xe)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:130 +0xde fp=0x12757f98 sp=0x12757f80
runtime.goparkunlock(0xb79e60, 0x9079a8, 0xe)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:136 +0x42 fp=0x12757fac sp=0x12757f98
runtime.runfinq()
/usr/local/Cellar/go/1.4/libexec/src/runtime/malloc.go:727 +0x9c fp=0x12757ff0 sp=0x12757fac
runtime.goexit()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:2287 +0x1 fp=0x12757ff4 sp=0x12757ff0
created by runtime.createfing
/usr/local/Cellar/go/1.4/libexec/src/runtime/malloc.go:707 +0x59
PS C:\dev\hugo\micro.moda> hugo server --watch | Out-File c:\dev\hugo\startup.txt
runtime: garbage collector found invalid heap pointer *(0xb705a0+0x12ac)=0x42cd0000 s=nil
fatal error: invalid heap pointer

runtime stack:
runtime.throw(0xb5e403)
/usr/local/Cellar/go/1.4/libexec/src/runtime/panic.go:491 +0x83 fp=0x3efce4 sp=0x3efccc
scanblock(0xb705a0, 0x9040, 0x2dc708)
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:378 +0x48d fp=0x3efd84 sp=0x3efce4
markroot(0x1274c0a0, 0x1)
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:496 +0x139 fp=0x3efdbc sp=0x3efd84
runtime.parfordo(0x1274c0a0)
/usr/local/Cellar/go/1.4/libexec/src/runtime/parfor.c:91 +0x115 fp=0x3efe18 sp=0x3efdbc
gc(0x3eff54)
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:1439 +0x1fb fp=0x3eff44 sp=0x3efe18
runtime.gc_m()
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:1368 +0xd2 fp=0x3eff64 sp=0x3eff44
runtime.onM(0x12752000)
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:266 +0x50 fp=0x3eff68 sp=0x3eff64
runtime.mstart()
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.c:818 fp=0x3eff6c sp=0x3eff68

goroutine 1 [garbage collection]:
runtime.switchtoM()

– truncated here –

goroutine 2 [force gc (idle)]:
runtime.gopark(0x42c540, 0xb71358, 0x907de8, 0xf)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:130 +0xde fp=0x1275afcc sp=0x1275afb4
runtime.goparkunlock(0xb71358, 0x907de8, 0xf)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:136 +0x42 fp=0x1275afe0 sp=0x1275afcc
runtime.forcegchelper()
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:99 +0xa3 fp=0x1275aff0 sp=0x1275afe0
runtime.goexit()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:2287 +0x1 fp=0x1275aff4 sp=0x1275aff0
created by runtime.init┬╖4
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:87 +0x25

goroutine 3 [GC sweep wait]:
runtime.gopark(0x42c540, 0xb79e64, 0x9006c8, 0xd)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:130 +0xde fp=0x1275cfcc sp=0x1275cfb4
runtime.goparkunlock(0xb79e64, 0x9006c8, 0xd)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:136 +0x42 fp=0x1275cfe0 sp=0x1275cfcc
runtime.bgsweep()
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.go:98 +0x97 fp=0x1275cff0 sp=0x1275cfe0
runtime.goexit()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:2287 +0x1 fp=0x1275cff4 sp=0x1275cff0
created by gc
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:1383

goroutine 4 [finalizer wait]:
runtime.gopark(0x42c540, 0xb79e60, 0x9079a8, 0xe)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:130 +0xde fp=0x12757f98 sp=0x12757f80
runtime.goparkunlock(0xb79e60, 0x9079a8, 0xe)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:136 +0x42 fp=0x12757fac sp=0x12757f98
runtime.runfinq()
/usr/local/Cellar/go/1.4/libexec/src/runtime/malloc.go:727 +0x9c fp=0x12757ff0 sp=0x12757fac
runtime.goexit()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:2287 +0x1 fp=0x12757ff4 sp=0x12757ff0
created by runtime.createfing
/usr/local/Cellar/go/1.4/libexec/src/runtime/malloc.go:707 +0x59
PS C:\dev\hugo\micro.moda> hugo server | Out-File c:\dev\hugo\startup.txt
runtime: garbage collector found invalid heap pointer *(0xb705a0+0x12ac)=0x42cc0000 s=nil
fatal error: invalid heap pointer

runtime stack:
runtime.throw(0xb5e403)
/usr/local/Cellar/go/1.4/libexec/src/runtime/panic.go:491 +0x83 fp=0xcfc58 sp=0xcfc40
scanblock(0xb705a0, 0x9040, 0x18c708)
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:378 +0x48d fp=0xcfcf8 sp=0xcfc58
markroot(0x1283c0a0, 0x1)
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:496 +0x139 fp=0xcfd30 sp=0xcfcf8
runtime.parfordo(0x1283c0a0)
/usr/local/Cellar/go/1.4/libexec/src/runtime/parfor.c:91 +0x115 fp=0xcfd8c sp=0xcfd30
gc(0xcfec8)
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:1439 +0x1fb fp=0xcfeb8 sp=0xcfd8c
runtime.gc_m()
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:1368 +0xd2 fp=0xcfed8 sp=0xcfeb8
runtime.onM(0xb71878)
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:266 +0x50 fp=0xcfedc sp=0xcfed8
runtime.mstart()
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.c:818 fp=0xcfee0 sp=0xcfedc

goroutine 1 [garbage collection]:
runtime.switchtoM()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:208 fp=0x128870e8 sp=0x128870e4
runtime.gogc(0x0)
/usr/local/Cellar/go/1.4/libexec/src/runtime/malloc.go:469 +0x1aa fp=0x12887108 sp=0x128870e8
runtime.mallocgc(0x10, 0x0, 0x3, 0x8)
/usr/local/Cellar/go/1.4/libexec/src/runtime/malloc.go:341 +0x2c4 fp=0x12887160 sp=0x12887108
runtime.rawstring(0xc, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/runtime/string.go:195 +0x88 fp=0x12887178 sp=0x12887160
runtime.slicebytetostring(0x4448086d, 0xc, 0x74358, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/runtime/string.go:69 +0x38 fp=0x128871c0 sp=0x12887178
encoding/json.unquote(0x4448086d, 0xc, 0x74358, 0x0, 0x0, 0x1)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:968 +0x79 fp=0x128871e0 sp=0x128871c0
– truncated here –
/Users/spf13/gopath/src/github.com/spf13/hugo/parser/frontmatter.go:189 +0x81 fp=0x128879dc sp=0x128879b0
github.com/spf13/hugo/hugolib.readData(0x128ffbc0, 0x0, 0x0, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/hugolib/site.go:321 +0xf6 fp=0x12887a3c sp=0x128879dc
github.com/spf13/hugo/hugolib.(*Site).loadData(0x128e9500, 0x12887bfc, 0x1, 0x2, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/hugolib/site.go:287 +0x35c fp=0x12887b9c sp=0x12887a3c
github.com/spf13/hugo/hugolib.(*Site).Process(0x128e9500, 0x18e040, 0x12966d68)
/Users/spf13/gopath/src/github.com/spf13/hugo/hugolib/site.go:347 +0x33f fp=0x12887c10 sp=0x12887b9c
github.com/spf13/hugo/hugolib.(*Site).Build(0x128e9500, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/hugolib/site.go:241 +0x38 fp=0x12887c7c sp=0x12887c10
github.com/spf13/hugo/commands.buildSite(0x12887d13, 0x1, 0x1, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/commands/hugo.go:342 +0xb3 fp=0x12887cf4 sp=0x12887c7c
github.com/spf13/hugo/commands.build(0x12887dae, 0x1, 0x1)
/Users/spf13/gopath/src/github.com/spf13/hugo/commands/hugo.go:254 +0x216 fp=0x12887d8c sp=0x12887cf4
github.com/spf13/hugo/commands.server(0xb6dd40, 0xb79ebc, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/commands/server.go:97 +0x43d fp=0x12887e58 sp=0x12887d8c
github.com/spf13/cobra.(*Command).execute(0xb6dd40, 0xb79ebc, 0x0, 0x0, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/cobra/command.go:399 +0x2a1 fp=0x12887eb4 sp=0x12887e58
github.com/spf13/cobra.(*Command).findAndExecute(0xb6d840, 0x128380f8, 0x1, 0x1, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/cobra/command.go:366 +0xa1 fp=0x12887ef4 sp=0x12887eb4
github.com/spf13/cobra.(*Command).Execute(0xb6d840, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/cobra/command.go:452 +0x83a fp=0x12887fa0 sp=0x12887ef4
github.com/spf13/hugo/commands.Execute()
/Users/spf13/gopath/src/github.com/spf13/hugo/commands/hugo.go:64 +0x2f fp=0x12887fc0 sp=0x12887fa0
main.main()
/Users/spf13/gopath/src/github.com/spf13/hugo/main.go:24 +0x2f fp=0x12887fcc sp=0x12887fc0
runtime.main()
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:63 +0xcc fp=0x12887ff0 sp=0x12887fcc
runtime.goexit()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:2287 +0x1 fp=0x12887ff4 sp=0x12887ff0

goroutine 2 [force gc (idle)]:
runtime.gopark(0x42c540, 0xb71358, 0x907de8, 0xf)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:130 +0xde fp=0x1284afcc sp=0x1284afb4
runtime.goparkunlock(0xb71358, 0x907de8, 0xf)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:136 +0x42 fp=0x1284afe0 sp=0x1284afcc
runtime.forcegchelper()
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:99 +0xa3 fp=0x1284aff0 sp=0x1284afe0
runtime.goexit()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:2287 +0x1 fp=0x1284aff4 sp=0x1284aff0
created by runtime.init┬╖4
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:87 +0x25

goroutine 3 [GC sweep wait]:
runtime.gopark(0x42c540, 0xb79e64, 0x9006c8, 0xd)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:130 +0xde fp=0x1284cfcc sp=0x1284cfb4
runtime.goparkunlock(0xb79e64, 0x9006c8, 0xd)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:136 +0x42 fp=0x1284cfe0 sp=0x1284cfcc
runtime.bgsweep()
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.go:98 +0x97 fp=0x1284cff0 sp=0x1284cfe0
runtime.goexit()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:2287 +0x1 fp=0x1284cff4 sp=0x1284cff0
created by gc
/usr/local/Cellar/go/1.4/libexec/src/runtime/mgc0.c:1383

goroutine 4 [finalizer wait]:
runtime.gopark(0x42c540, 0xb79e60, 0x9079a8, 0xe)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:130 +0xde fp=0x12847f98 sp=0x12847f80
runtime.goparkunlock(0xb79e60, 0x9079a8, 0xe)
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:136 +0x42 fp=0x12847fac sp=0x12847f98
runtime.runfinq()
/usr/local/Cellar/go/1.4/libexec/src/runtime/malloc.go:727 +0x9c fp=0x12847ff0 sp=0x12847fac
runtime.goexit()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:2287 +0x1 fp=0x12847ff4 sp=0x12847ff0
created by runtime.createfing
/usr/local/Cellar/go/1.4/libexec/src/runtime/malloc.go:707 +0x59

2

No, you’re not throwing too much at it, this looks like a Go-bug.

  • What Hugo version?
  • What OS? Windows 64 bit?

I would start by upgrading to latest Go release – 1.4.2. There are some related fixes in 1.4.1 and 1.4.2.

3

On a technical note, here is Go’s Russ Cox about this:

https://groups.google.com/forum/#!search/invalid$20heap$20pointer$20json/golang-dev/HJKnsc6p8DI/1d6nrJH5920J

In this case it looks like an integer (1390838824) is treated as a pointer. By the json package, probably?

1 Like
4

I’m using go 1.4.2 windows/386. Hugo is on v0.13 BuildDate: 2015-03-31T22:25:02-06:00. I’ll try to take a look at the Google Groups link you sent later tonight to see if I can find a work around. Thanks!

5

It would also help if you could reproduce this with a smaller project and share it. My guess this has nothing to do with the number of files, but the content …

Maybe something in the JSON files in /data.

6

Ya, I started out with three of so of the files while testing and all looked good. I’ll try to scale back and identify the point of failure, but with so many files, that will take a while. Will circle back and advise if I find anything interesting. Thanks again!

7

One more thought. I haven’t investigated verbose logging or step analysis yet. Might I get a hint from it as to where to look? If that sounds like a fools errand (knowing the problem is higher in the stack), I’ll just go back, scale down and iterate until I find more info.

8

Verbose logging will not help here (you can add -v to hugo, but it’s not verbose enough for your case)…

I would start searching for the value 1390838824 in the /data folder.

9

Hmm. Well, the problem is definitely in the data directory. If I rename it to odata, it worked great. From there, I just started copying files over to see if I could identify the point of failure. I couldn’t find the value you were referencing, but I think I have reached some upper bound threshold for data. After playing around with various permutations for files in data, I’ve found that I run across the problem when I reach 286MB of json spanning 118 files.

I’m sure the file quantity is too small to be involved, so I’m guessing it’s related to the size of the files themselves (admitting large). That said, I was kinda hoping it would just be slower to generate than something that would fail. I can send 'ya sample data if you think it would be helpful, but for now, I’m thinking I’ll just step back and look at ways to reduce the size of the json to see if I can find some middle ground that works with the current setup.

10

Well, I radically reduced the file size of the json files I was using, but I’m guessing not enough. If I radically reduce the quantity of files, I can get it to start, but not with all of 'em in there.

Here’s an example of what I’m getting now with all of them in there …

Any insights from the updated error?

PS C:\dev\hugo\micro.moda> hugo server
fatal error: runtime: cannot map pages in arena address space

runtime stack:
runtime.SysMap(0x0, 0x100000, 0x24fe01, 0xb80718)
/usr/local/Cellar/go/1.4/libexec/src/runtime/mem_windows.c:131 +0x7f
runtime.MHeap_SysAlloc(0xb83500, 0x100000, 0x631c6940)
/usr/local/Cellar/go/1.4/libexec/src/runtime/malloc.c:284 +0xf1
runtime.MHeap_Alloc(0xb83500, 0x1, 0x4, 0x100, 0x40a3c8)
/usr/local/Cellar/go/1.4/libexec/src/runtime/mheap.c:240 +0x66
runtime.MCentral_CacheSpan(0xb874b4, 0x631c6940)
/usr/local/Cellar/go/1.4/libexec/src/runtime/mcentral.c:85 +0x122
runtime.MCache_Refill(0x180000, 0x4, 0x631c6940)
/usr/local/Cellar/go/1.4/libexec/src/runtime/mcache.c:90 +0x83

goroutine 1 [running]:
runtime.switchtoM()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:208 fp=0x129a3254 sp=0x129a3250
runtime.mallocgc(0x30, 0x0, 0x3, 0x2e)
/usr/local/Cellar/go/1.4/libexec/src/runtime/malloc.go:178 +0x68e fp=0x129a32ac sp=0x129a3254
runtime.rawstring(0x24, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/runtime/string.go:195 +0x88 fp=0x129a32c4 sp=0x129a32ac
runtime.slicebytetostring(0x73379fb0, 0x24, 0x2e, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/runtime/string.go:69 +0x38 fp=0x129a330c sp=0x129a32c4
encoding/json.unquote(0x73379fb0, 0x24, 0x2e, 0x0, 0x0, 0x7337ae01)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:968 +0x79 fp=0x129a332c sp=0x129a330c
encoding/json.(*decodeState).literalInterface(0x72fcb400, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:933 +0xee fp=0x129a3388 sp=0x129a332c
encoding/json.(*decodeState).valueInterface(0x72fcb400, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:836 +0x55 fp=0x129a33b0 sp=0x129a3388
encoding/json.(*decodeState).objectInterface(0x72fcb400, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x129a33fc sp=0x129a33b0
encoding/json.(*decodeState).valueInterface(0x72fcb400, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x129a3424 sp=0x129a33fc
encoding/json.(*decodeState).objectInterface(0x72fcb400, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x129a3470 sp=0x129a3424
encoding/json.(*decodeState).valueInterface(0x72fcb400, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x129a3498 sp=0x129a3470
encoding/json.(*decodeState).objectInterface(0x72fcb400, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x129a34e4 sp=0x129a3498
encoding/json.(*decodeState).valueInterface(0x72fcb400, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x129a350c sp=0x129a34e4
encoding/json.(*decodeState).objectInterface(0x72fcb400, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x129a3558 sp=0x129a350c
encoding/json.(*decodeState).valueInterface(0x72fcb400, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x129a3580 sp=0x129a3558
encoding/json.(*decodeState).objectInterface(0x72fcb400, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x129a35cc sp=0x129a3580
encoding/json.(*decodeState).valueInterface(0x72fcb400, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x129a35f4 sp=0x129a35cc
encoding/json.(*decodeState).objectInterface(0x72fcb400, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x129a3640 sp=0x129a35f4
encoding/json.(*decodeState).valueInterface(0x72fcb400, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x129a3668 sp=0x129a3640
encoding/json.(*decodeState).objectInterface(0x72fcb400, 0x9)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x129a36b4 sp=0x129a3668
encoding/json.(*decodeState).valueInterface(0x72fcb400, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:834 +0x76 fp=0x129a36dc sp=0x129a36b4
encoding/json.(*decodeState).objectInterface(0x72fcb400, 0x73159570)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:900 +0x1a2 fp=0x129a3728 sp=0x129a36dc
encoding/json.(*decodeState).object(0x72fcb400, 0x7dbd80, 0x73159570, 0xd4)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:497 +0x306 fp=0x129a38c8 sp=0x129a3728
encoding/json.(*decodeState).value(0x72fcb400, 0x7c5d80, 0x73159570, 0x16)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:288 +0x2cd fp=0x129a3928 sp=0x129a38c8
encoding/json.(*decodeState).unmarshal(0x72fcb400, 0x7c5d80, 0x73159570, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:149 +0x177 fp=0x129a3978 sp=0x129a3928
encoding/json.Unmarshal(0x73188000, 0x80643, 0x80643, 0x7c5d80, 0x73159570, 0x0, 0x0)
/usr/local/Cellar/go/1.4/libexec/src/encoding/json/decode.go:78 +0xf7 fp=0x129a39b0 sp=0x129a3978
github.com/spf13/hugo/parser.HandleJsonMetaData(0x73188000, 0x80643, 0x80643, 0x0, 0x0, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/parser/frontmatter.go:189 +0x81 fp=0x129a39dc sp=0x129a39b0
github.com/spf13/hugo/hugolib.readData(0x1b4fc240, 0x0, 0x0, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/hugolib/site.go:321 +0xf6 fp=0x129a3a3c sp=0x129a39dc
github.com/spf13/hugo/hugolib.(*Site).loadData(0x12a59d00, 0x129a3bfc, 0x1, 0x2, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/hugolib/site.go:287 +0x35c fp=0x129a3b9c sp=0x129a3a3c
github.com/spf13/hugo/hugolib.(*Site).Process(0x12a59d00, 0x18e040, 0x1293f5a0)
/Users/spf13/gopath/src/github.com/spf13/hugo/hugolib/site.go:347 +0x33f fp=0x129a3c10 sp=0x129a3b9c
github.com/spf13/hugo/hugolib.(*Site).Build(0x12a59d00, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/hugolib/site.go:241 +0x38 fp=0x129a3c7c sp=0x129a3c10
github.com/spf13/hugo/commands.buildSite(0x129a3d13, 0x1, 0x1, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/commands/hugo.go:342 +0xb3 fp=0x129a3cf4 sp=0x129a3c7c
github.com/spf13/hugo/commands.build(0x129a3dae, 0x1, 0x1)
/Users/spf13/gopath/src/github.com/spf13/hugo/commands/hugo.go:254 +0x216 fp=0x129a3d8c sp=0x129a3cf4
github.com/spf13/hugo/commands.server(0xb6dd40, 0xb79ebc, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/hugo/commands/server.go:97 +0x43d fp=0x129a3e58 sp=0x129a3d8c
github.com/spf13/cobra.(*Command).execute(0xb6dd40, 0xb79ebc, 0x0, 0x0, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/cobra/command.go:399 +0x2a1 fp=0x129a3eb4 sp=0x129a3e58
github.com/spf13/cobra.(*Command).findAndExecute(0xb6d840, 0x129280f8, 0x1, 0x1, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/cobra/command.go:366 +0xa1 fp=0x129a3ef4 sp=0x129a3eb4
github.com/spf13/cobra.(*Command).Execute(0xb6d840, 0x0, 0x0)
/Users/spf13/gopath/src/github.com/spf13/cobra/command.go:452 +0x83a fp=0x129a3fa0 sp=0x129a3ef4
github.com/spf13/hugo/commands.Execute()
/Users/spf13/gopath/src/github.com/spf13/hugo/commands/hugo.go:64 +0x2f fp=0x129a3fc0 sp=0x129a3fa0
main.main()
/Users/spf13/gopath/src/github.com/spf13/hugo/main.go:24 +0x2f fp=0x129a3fcc sp=0x129a3fc0
runtime.main()
/usr/local/Cellar/go/1.4/libexec/src/runtime/proc.go:63 +0xcc fp=0x129a3ff0 sp=0x129a3fcc
runtime.goexit()
/usr/local/Cellar/go/1.4/libexec/src/runtime/asm_386.s:2287 +0x1 fp=0x129a3ff4 sp=0x129a3ff0

11

The data files is loaded into memory as one big map (we may have to rethink that strategy), seems like you’re hitting some corner cases on Windows.

I notice you run the 32 bits Go version – I would, if possible, go with the 64 bit one.

I might look into this later today and test with some massive dummy data.

12

Did some quick tests on my Linux box with some 17,5 MB big json files – and the memory overhead is very big:

1 file: 2.04u 0.22s 1.67r 210864kB hugo
2 files: 3.31u 0.24s 2.67r 361008kB hugo

That’s 150 MB to represent a 17,5 MB file. Hmm …

See:

13

Roger that. Thanks for the confirmation! I did switch over to the 64 bit version, but got the same error that I’d initially reported with it. I’m happy to test other techniques if/when they’re ready.

Thanks again for all the follow-up.

14

I’ve been able to shrink the files down to get something workable. It takes around three minutes to generate the site. I hope to get the code up on github this weekend, so you can take a peek if you’re interested. In the meantime, I was wondering if you had any thoughts on whether or not I might have better luck if I switched to YAML or TOML for the format? I imagine it has something to do with the way data files are handled in general, but on the chance it is something unique to JSON, I thought I’d check.

Thanks again,
Matt

15

YAML and TOML may be worth a try, all have very different code paths. I might test it myself later today. I haven’t looked more closely into this one.

16

I haven’t tried substituting YAML or TOML for JSON yet, but I have got the code up on Github now.

The code is here: https://github.com/mattpaz/micro.moda

The work-in-progress site is available here: http://micro.moda/

I’m still getting up-to-speed with Hugo, but if you have any thoughts/ideas for optimization, I’m all ears.

Thanks again,

Matt