<p dir="auto">I am getting this mysterious error message after my program runs for 2-3 hours (full stack trace below). The program runs on tons of data as a part of my academic research, so unfortunately I can't provide the data for you to reproduce it.</p>
<p dir="auto">The code is here:<br>
<a href="https://github.com/fluhus/prices/tree/master/src/items2">https://github.com/fluhus/prices/tree/master/src/items2</a><br>
(sorry for scrappiness; It's in the middle of a rework)</p>
<p dir="auto">More info</p>
<ul dir="auto">
<li>See thread on golang nuts here: <a href="https://groups.google.com/forum/#!topic/golang-nuts/yn6_MDRFGr4" rel="nofollow">https://groups.google.com/forum/#!topic/golang-nuts/yn6_MDRFGr4</a></li>
<li>golang 1.5.1 linux/amd64</li>
<li>Machine runs Linux 64 bit with 32 cores and 64 GB RAM.</li>
<li>No cgo in my code.</li>
<li>No findings with race detector.</li>
</ul>
<p dir="auto">Let me know if you need additional information.</p>
<p dir="auto">Thanks a lot!</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="runtime:objectstart Span weird: p=0xc900000000 k=0x6480000 s.start=0xc8fffe8000 s.limit=0xc900001fe0 s.state=2
fatal error: objectstart: bad pointer in unexpected span

runtime stack:
runtime.throw(0x65c4c0, 0x2b)
        /usr/local/go/src/runtime/panic.go:527 +0x90
runtime.heapBitsForObject(0xc900000000, 0x0, 0x0, 0xc800000000, 0x7fad6c9123e0)
        /usr/local/go/src/runtime/mbitmap.go:217 +0x287
runtime.scanobject(0xc8ccd33b00, 0xc820036720)
        /usr/local/go/src/runtime/mgcmark.go:878 +0x239
runtime.gcDrainN(0xc820036720, 0x485e5)
        /usr/local/go/src/runtime/mgcmark.go:779 +0x8e
runtime.gcAssistAlloc.func1()
        /usr/local/go/src/runtime/mgcmark.go:232 +0x172
runtime.systemstack(0xc820022a00)
        /usr/local/go/src/runtime/asm_amd64.s:262 +0x79
runtime.mstart()
        /usr/local/go/src/runtime/proc1.go:674"><pre class="notranslate"><code class="notranslate">runtime:objectstart Span weird: p=0xc900000000 k=0x6480000 s.start=0xc8fffe8000 s.limit=0xc900001fe0 s.state=2
fatal error: objectstart: bad pointer in unexpected span

runtime stack:
runtime.throw(0x65c4c0, 0x2b)
        /usr/local/go/src/runtime/panic.go:527 +0x90
runtime.heapBitsForObject(0xc900000000, 0x0, 0x0, 0xc800000000, 0x7fad6c9123e0)
        /usr/local/go/src/runtime/mbitmap.go:217 +0x287
runtime.scanobject(0xc8ccd33b00, 0xc820036720)
        /usr/local/go/src/runtime/mgcmark.go:878 +0x239
runtime.gcDrainN(0xc820036720, 0x485e5)
        /usr/local/go/src/runtime/mgcmark.go:779 +0x8e
runtime.gcAssistAlloc.func1()
        /usr/local/go/src/runtime/mgcmark.go:232 +0x172
runtime.systemstack(0xc820022a00)
        /usr/local/go/src/runtime/asm_amd64.s:262 +0x79
runtime.mstart()
        /usr/local/go/src/runtime/proc1.go:674
</code></pre></div>
<p dir="auto">The following program crashes when run with <code class="notranslate">GODEBUG=invalidptr=1 GOARCH=386</code>:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="package main

import (
    &quot;math/rand&quot;
    &quot;runtime&quot;
)

func main() {
    P := 2 * runtime.GOMAXPROCS(0)
    ptrs := make([][]*int, P)
    for i := 0; i &lt; P; i++ {
        i := i
        go func() {
            r := rand.New(rand.NewSource(int64(i)))
            var sink []byte
            p := make([]*int, 4)
            z := make([]*int, 4)
            ptrs[i] = p
            for {
                if r.Intn(20) != 0 {
                    a := p[:1]
                    a[0] = new(int)
                    for j := range a {
                        a[j] = nil
                    }
                    for j := range p {
                        p[j] = new(int)
                    }
                    copy(p[:3], z)
                    for j := range p {
                        p[j] = new(int)
                    }
                    copy(p[1:4], z)
                    for j := range p {
                        p[j] = new(int)
                        copy(p[j:j+1], z)
                    }
                } else {
                    sink = make([]byte, 8&lt;&lt;10)
                }
            }
            _ = sink
        }()
    }
    select {}
}"><pre class="notranslate"><code class="notranslate">package main

import (
    "math/rand"
    "runtime"
)

func main() {
    P := 2 * runtime.GOMAXPROCS(0)
    ptrs := make([][]*int, P)
    for i := 0; i &lt; P; i++ {
        i := i
        go func() {
            r := rand.New(rand.NewSource(int64(i)))
            var sink []byte
            p := make([]*int, 4)
            z := make([]*int, 4)
            ptrs[i] = p
            for {
                if r.Intn(20) != 0 {
                    a := p[:1]
                    a[0] = new(int)
                    for j := range a {
                        a[j] = nil
                    }
                    for j := range p {
                        p[j] = new(int)
                    }
                    copy(p[:3], z)
                    for j := range p {
                        p[j] = new(int)
                    }
                    copy(p[1:4], z)
                    for j := range p {
                        p[j] = new(int)
                        copy(p[j:j+1], z)
                    }
                } else {
                    sink = make([]byte, 8&lt;&lt;10)
                }
            }
            _ = sink
        }()
    }
    select {}
}
</code></pre></div>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="GODEBUG=invalidptr=1 GOARCH=386 go run memclr.go

runtime: pointer 0x18d90000 to unallocated spanidx=0x548 span.start=0x185d2000 span.limit=0x185d4000 span.state=3
runtime: found in object at *(0x18456000+0x0)
object=0x18456000 k=0xc22b s.start*_PageSize=0x18456000 s.limit=0x18458000 s.sizeclass=2 s.elemsize=16
 *(object+0) = 0x0 &lt;==
 *(object+4) = 0x0
 *(object+8) = 0x18d9b354
 *(object+12) = 0x18d9b3b0
fatal error: found bad pointer in Go heap (incorrect use of unsafe or cgo?)"><pre class="notranslate"><code class="notranslate">GODEBUG=invalidptr=1 GOARCH=386 go run memclr.go

runtime: pointer 0x18d90000 to unallocated spanidx=0x548 span.start=0x185d2000 span.limit=0x185d4000 span.state=3
runtime: found in object at *(0x18456000+0x0)
object=0x18456000 k=0xc22b s.start*_PageSize=0x18456000 s.limit=0x18458000 s.sizeclass=2 s.elemsize=16
 *(object+0) = 0x0 &lt;==
 *(object+4) = 0x0
 *(object+8) = 0x18d9b354
 *(object+12) = 0x18d9b3b0
fatal error: found bad pointer in Go heap (incorrect use of unsafe or cgo?)
</code></pre></div>
<p dir="auto">The problem is that memclr clears 4-byte regions on 386 with 2 separate 2-byte stores:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="_3or4:
    MOVW    AX, (DI)
    MOVW    AX, -2(DI)(BX*1)"><pre class="notranslate"><code class="notranslate">_3or4:
    MOVW    AX, (DI)
    MOVW    AX, -2(DI)(BX*1)
</code></pre></div>
<p dir="auto">This exposes corrupted pointer to GC.</p>
<p dir="auto">Not sure whether it can crash without invalidptr, I would not exclude such possibility. But it's clearly bad and can lead to false retention. Also it makes data race failure modes significantly worse, introducing exploitation possibilities like use-after-free and easy controlled overwrite.</p>
<p dir="auto">There is similar issue in amd64p32 memclr which clears pointers with byte stores:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="TEXT runtime·memclr(SB),NOSPLIT,$0-8
    MOVL    ptr+0(FP), DI
    MOVL    n+4(FP), CX
    MOVQ    CX, BX
    ANDQ    $3, BX
    SHRQ    $2, CX
    MOVQ    $0, AX
    CLD
    REP
    STOSL
    MOVQ    BX, CX
    REP
    STOSB
    RET"><pre class="notranslate"><code class="notranslate">TEXT runtime·memclr(SB),NOSPLIT,$0-8
    MOVL    ptr+0(FP), DI
    MOVL    n+4(FP), CX
    MOVQ    CX, BX
    ANDQ    $3, BX
    SHRQ    $2, CX
    MOVQ    $0, AX
    CLD
    REP
    STOSL
    MOVQ    BX, CX
    REP
    STOSB
    RET
</code></pre></div>
<p dir="auto">Similar issue in 386 memmove which moves pointer with 2 stores:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="move_3or4:
    MOVW    (SI), AX
    MOVW    -2(SI)(BX*1), CX
    MOVW    AX, (DI)
    MOVW    CX, -2(DI)(BX*1)
    RET"><pre class="notranslate"><code class="notranslate">move_3or4:
    MOVW    (SI), AX
    MOVW    -2(SI)(BX*1), CX
    MOVW    AX, (DI)
    MOVW    CX, -2(DI)(BX*1)
    RET
</code></pre></div>
<p dir="auto">Similar issue in plan9/386 memclr:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="_1or2:
    MOVB    AX, (DI)
    MOVB    AX, -1(DI)(BX*1)"><pre class="notranslate"><code class="notranslate">_1or2:
    MOVB    AX, (DI)
    MOVB    AX, -1(DI)(BX*1)
</code></pre></div>
<p dir="auto">Similar issue in amd64 memclr:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="_5through8:
    MOVL    AX, (DI)
    MOVL    AX, -4(DI)(BX*1)"><pre class="notranslate"><code class="notranslate">_5through8:
    MOVL    AX, (DI)
    MOVL    AX, -4(DI)(BX*1)
</code></pre></div>
<p dir="auto">Did not look at arm/ppc. I guess we just need to go through all of them one-by-one.<br>
<a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/rsc/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/rsc">@rsc</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/randall77/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/randall77">@randall77</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/RLH/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/RLH">@RLH</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/aclements/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/aclements">@aclements</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/dr2chase/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/dr2chase">@dr2chase</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/davecheney/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/davecheney">@davecheney</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/minux/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/minux">@minux</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/0intro/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/0intro">@0intro</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/4ad/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/4ad">@4ad</a></p>