title: Receiving "Illegal Instruction" when executing glibc's ld-2.6.1.so
body: <p><strong>Background</strong></p>
  
  <p>"OS": Stripped down Linux, very customized, no internet access (no yum, apt-get, etc)</p>
  
  <p>Kernel: 2.6.19.1</p>
  
  <p>Target: 32-bit, armv5te</p>
  
  <p>Current LibC: 2.3.6</p>
  
  <p>Target LibC: 2.6.1</p>
  
  <p><strong>Issue</strong></p>
  
  <p>Received an <code>.ipk</code> from a 3rd party vendor containing an updated version of <code>glibc</code>. Started by investigating the compatibility of the shared objects contained within the <code>.ipk</code> package by placing them on the target platform and attempting to run the <code>ld-2.6.1.so</code> directly (chose this library because my understanding is that it has no dynamic linking to other objects). </p>
  
  <p>Running this shared object library directly results in "Illegal Instruction". My initial thought was that <code>ld</code> was built for the wrong architecture, however, review of the <code>readelf</code> output appears to indicate it was set up correctly:</p>
  
  <pre><code>[root@blg_g34_z2_03 lib]# readelf -a ld-2.6.1.so 
  ELF Header:
    Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00 
    Class:                             ELF32
    Data:                              2's complement, little endian
    Version:                           1 (current)
    OS/ABI:                            UNIX - System V
    ABI Version:                       0
    Type:                              DYN (Shared object file)
    Machine:                           ARM
    Version:                           0x1
    Entry point address:               0x800
    Start of program headers:          52 (bytes into file)
    Start of section headers:          116488 (bytes into file)
    Flags:                             0x4000002, has entry point, Version4 EABI
    Size of this header:               52 (bytes)
    Size of program headers:           32 (bytes)
    Number of program headers:         6
    Size of section headers:           40 (bytes)
    Number of section headers:         26
    Section header string table index: 25
  
  Section Headers:
    [Nr] Name              Type            Addr     Off    Size   ES Flg Lk Inf Al
    [ 0]                   NULL            00000000 000000 000000 00      0   0  0
    [ 1] .hash             HASH            000000f4 0000f4 0000c4 04   A  3   0  4
    [ 2] .gnu.hash         GNU_HASH        000001b8 0001b8 0000e0 04   A  3   0  4
    [ 3] .dynsym           DYNSYM          00000298 000298 0001e0 10   A  4   3  4
    [ 4] .dynstr           STRTAB          00000478 000478 0001ac 00   A  0   0  1
    [ 5] .gnu.version      VERSYM          00000624 000624 00003c 02   A  3   0  2
    [ 6] .gnu.version_d    VERDEF          00000660 000660 00005c 00   A  4   3  4
    [ 7] .rel.dyn          REL             000006bc 0006bc 0000b8 08   A  3   0  4
    [ 8] .rel.plt          REL             00000774 000774 000030 08   A  3   9  4
    [ 9] .plt              PROGBITS        000007a4 0007a4 00005c 04  AX  0   0  4
    [10] .text             PROGBITS        00000800 000800 017324 00  AX  0   0 16
    [11] __libc_freeres_fn PROGBITS        00017b24 017b24 000148 00  AX  0   0  4
    [12] .rodata           PROGBITS        00017c6c 017c6c 003828 00   A  0   0  4
    [13] .ARM.extab        PROGBITS        0001b494 01b494 000048 00   A  0   0  4
    [14] .ARM.exidx        ARM_EXIDX       0001b4dc 01b4dc 000078 00  AL 10   0  4
    [15] .eh_frame_hdr     PROGBITS        0001b554 01b554 00001c 00   A  0   0  4
    [16] .eh_frame         PROGBITS        0001b570 01b570 00007c 00   A  0   0  4
    [17] .data.rel.ro      PROGBITS        00023db0 01bdb0 000194 00  WA  0   0  8
    [18] .dynamic          DYNAMIC         00023f44 01bf44 0000b8 08  WA  4   0  4
    [19] .got              PROGBITS        00024000 01c000 00005c 04  WA  0   0  4
    [20] .data             PROGBITS        00024060 01c060 000580 00  WA  0   0  8
    [21] __libc_subfreeres PROGBITS        000245e0 01c5e0 000004 00  WA  0   0  4
    [22] .bss              NOBITS          000245e4 01c5e4 0000e4 00  WA  0   0  4
    [23] .ARM.attributes   ARM_ATTRIBUTES  00000000 01c5e4 000019 00      0   0  1
    [24] .gnu_debuglink    PROGBITS        00000000 01c5fd 000010 00      0   0  1
    [25] .shstrtab         STRTAB          00000000 01c60d 0000f8 00      0   0  1
  Key to Flags:
    W (write), A (alloc), X (execute), M (merge), S (strings)
    I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
    O (extra OS processing required) o (OS specific), p (processor specific)
  
  There are no section groups in this file.
  
  Program Headers:
    Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
    EXIDX          0x01b4dc 0x0001b4dc 0x0001b4dc 0x00078 0x00078 R   0x4
    LOAD           0x000000 0x00000000 0x00000000 0x1b5ec 0x1b5ec R E 0x8000
    LOAD           0x01bdb0 0x00023db0 0x00023db0 0x00834 0x00918 RW  0x8000
    DYNAMIC        0x01bf44 0x00023f44 0x00023f44 0x000b8 0x000b8 RW  0x4
    GNU_EH_FRAME   0x01b554 0x0001b554 0x0001b554 0x0001c 0x0001c R   0x4
    GNU_RELRO      0x01bdb0 0x00023db0 0x00023db0 0x00250 0x00250 R   0x1
  
   Section to Segment mapping:
    Segment Sections...
     00     .ARM.exidx 
     01     .hash .gnu.hash .dynsym .dynstr .gnu.version .gnu.version_d .rel.dyn .rel.plt .plt .text __libc_freeres_fn .rodata .ARM.extab .ARM.exidx .eh_frame_hdr .eh_frame 
     02     .data.rel.ro .dynamic .got .data __libc_subfreeres .bss 
     03     .dynamic 
     04     .eh_frame_hdr 
     05     .data.rel.ro .dynamic 
  
  Dynamic section at offset 0x1bf44 contains 19 entries:
    Tag        Type                         Name/Value
   0x0000000e (SONAME)                     Library soname: [ld-linux.so.3]
   0x00000004 (HASH)                       0xf4
   0x6ffffef5 (GNU_HASH)                   0x1b8
   0x00000005 (STRTAB)                     0x478
   0x00000006 (SYMTAB)                     0x298
   0x0000000a (STRSZ)                      428 (bytes)
   0x0000000b (SYMENT)                     16 (bytes)
   0x00000003 (PLTGOT)                     0x24000
   0x00000002 (PLTRELSZ)                   48 (bytes)
   0x00000014 (PLTREL)                     REL
   0x00000017 (JMPREL)                     0x774
   0x00000011 (REL)                        0x6bc
   0x00000012 (RELSZ)                      184 (bytes)
   0x00000013 (RELENT)                     8 (bytes)
   0x6ffffffc (VERDEF)                     0x660
   0x6ffffffd (VERDEFNUM)                  3
   0x6ffffff0 (VERSYM)                     0x624
   0x6ffffffa (RELCOUNT)                   20
   0x00000000 (NULL)                       0x0
  
  Relocation section '.rel.dyn' at offset 0x6bc contains 23 entries:
   Offset     Info    Type            Sym.Value  Sym. Name
  00023e8c  00000017 R_ARM_RELATIVE   
  00023e90  00000017 R_ARM_RELATIVE   
  00023e94  00000017 R_ARM_RELATIVE   
  00023e98  00000017 R_ARM_RELATIVE   
  00023e9c  00000017 R_ARM_RELATIVE   
  00023ea0  00000017 R_ARM_RELATIVE   
  00023ea4  00000017 R_ARM_RELATIVE   
  00023ea8  00000017 R_ARM_RELATIVE   
  00024024  00000017 R_ARM_RELATIVE   
  00024028  00000017 R_ARM_RELATIVE   
  00024030  00000017 R_ARM_RELATIVE   
  00024038  00000017 R_ARM_RELATIVE   
  0002403c  00000017 R_ARM_RELATIVE   
  00024040  00000017 R_ARM_RELATIVE   
  00024044  00000017 R_ARM_RELATIVE   
  00024048  00000017 R_ARM_RELATIVE   
  0002404c  00000017 R_ARM_RELATIVE   
  00024050  00000017 R_ARM_RELATIVE   
  00024054  00000017 R_ARM_RELATIVE   
  000245e0  00000017 R_ARM_RELATIVE   
  0002402c  00001b15 R_ARM_GLOB_DAT    00023f0c   __stack_chk_guard
  00024034  00001815 R_ARM_GLOB_DAT    00014c84   malloc
  00024058  00000b15 R_ARM_GLOB_DAT    000246b4   _r_debug
  
  Relocation section '.rel.plt' at offset 0x774 contains 6 entries:
   Offset     Info    Type            Sym.Value  Sym. Name
  0002400c  00000e16 R_ARM_JUMP_SLOT   00014b50   __libc_memalign
  00024010  00001816 R_ARM_JUMP_SLOT   00014c84   malloc
  00024014  00001016 R_ARM_JUMP_SLOT   00014d38   calloc
  00024018  00000916 R_ARM_JUMP_SLOT   00014c90   realloc
  0002401c  00000716 R_ARM_JUMP_SLOT   000086d4   _dl_cache_libcmp
  00024020  00000816 R_ARM_JUMP_SLOT   00014b08   free
  
  Unwind table index '.ARM.exidx' at offset 0x1b4dc contains 15 entries:
  
  0x8ed4: 0x80b0b0b0
    Compact model index: 0
    0xb0      finish
    0xb0      finish
    0xb0      finish
  
  0x8f0c: 0x8000abb0
    Compact model index: 0
    0x00      vsp = vsp + 4
    0xab      pop {r4, r5, r6, r7, r14}
    0xb0      finish
  
  0x8ff0: 0x8000abb0
    Compact model index: 0
    0x00      vsp = vsp + 4
    0xab      pop {r4, r5, r6, r7, r14}
    0xb0      finish
  
  0x91b0: 0x800eafb0
    Compact model index: 0
    0x0e      vsp = vsp + 60
    0xaf      pop {r4, r5, r6, r7, r8, r9, r10, r11, r14}
    0xb0      finish
  
  0x9574: @0x1b494
    Compact model index: 1
    0x9b      vsp = r11
    0x49      vsp = vsp - 40
    0x86 0xff pop {r4, r5, r6, r7, r8, r9, r10, r11, r13, r14}
    0xb0      finish
    0xb0      finish
  
  0xcf44: 0x800aafb0
    Compact model index: 0
    0x0a      vsp = vsp + 44
    0xaf      pop {r4, r5, r6, r7, r8, r9, r10, r11, r14}
    0xb0      finish
  
  0xd060: 0x8014afb0
    Compact model index: 0
    0x14      vsp = vsp + 84
    0xaf      pop {r4, r5, r6, r7, r8, r9, r10, r11, r14}
    0xb0      finish
  
  0xd5c0: 0x8006afb0
    Compact model index: 0
    0x06      vsp = vsp + 28
    0xaf      pop {r4, r5, r6, r7, r8, r9, r10, r11, r14}
    0xb0      finish
  
  0x15380: @0x1b4a0
    Compact model index: 1
    0x01      vsp = vsp + 8
    0x80 0x08 pop {r7}
    0xb1 0x0e pop {r1, r2, r3}
    0xb0      finish
  
  0x158c0: @0x1b4ac
    Compact model index: 1
    0x02      vsp = vsp + 12
    0xb1 0x0f pop {r0, r1, r2, r3}
    0x8f 0xff pop {r4, r5, r6, r7, r8, r9, r10, r11, r12, r13, r14, r15}
    0xb0      finish
  
  0x158d8: @0x1b4b8
    Compact model index: 1
    0x07      vsp = vsp + 32
    0xb1 0x0f pop {r0, r1, r2, r3}
    0x8f 0xff pop {r4, r5, r6, r7, r8, r9, r10, r11, r12, r13, r14, r15}
    0xb0      finish
  
  0x158e8: @0x1b4c4
    Compact model index: 1
    0x29      vsp = vsp + 168
    0xb1 0x0f pop {r0, r1, r2, r3}
    0x8f 0xff pop {r4, r5, r6, r7, r8, r9, r10, r11, r12, r13, r14, r15}
    0xb0      finish
  
  0x158f8: @0x1b4d0
    Compact model index: 1
    0x27      vsp = vsp + 160
    0xb1 0x0f pop {r0, r1, r2, r3}
    0x8f 0xff pop {r4, r5, r6, r7, r8, r9, r10, r11, r12, r13, r14, r15}
    0xb0      finish
  
  0x1704c: 0x8004afb0
    Compact model index: 0
    0x04      vsp = vsp + 20
    0xaf      pop {r4, r5, r6, r7, r8, r9, r10, r11, r14}
    0xb0      finish
  
  0x17580: 0x8002afb0
    Compact model index: 0
    0x02      vsp = vsp + 12
    0xaf      pop {r4, r5, r6, r7, r8, r9, r10, r11, r14}
    0xb0      finish
  
  
  Symbol table '.dynsym' contains 30 entries:
     Num:    Value  Size Type    Bind   Vis      Ndx Name
       0: 00000000     0 NOTYPE  LOCAL  DEFAULT  UND 
       1: 00000800     0 SECTION LOCAL  DEFAULT   10 
       2: 00023db0     0 SECTION LOCAL  DEFAULT   17 
       3: 000103d4    44 FUNC    GLOBAL DEFAULT   10 _dl_get_tls_static_info@@GLIBC_PRIVATE
       4: 00023f10     4 OBJECT  GLOBAL DEFAULT   17 __pointer_chk_guard@@GLIBC_PRIVATE
       5: 00000000     0 OBJECT  GLOBAL DEFAULT  ABS GLIBC_PRIVATE
       6: 00000000     0 OBJECT  GLOBAL DEFAULT  ABS GLIBC_2.4
       7: 000086d4   260 FUNC    GLOBAL DEFAULT   10 _dl_cache_libcmp@@GLIBC_PRIVATE
       8: 00014b08    72 FUNC    WEAK   DEFAULT   10 free@@GLIBC_2.4
       9: 00014c90   168 FUNC    WEAK   DEFAULT   10 realloc@@GLIBC_2.4
      10: 00010ed8    40 FUNC    GLOBAL DEFAULT   10 _dl_allocate_tls@@GLIBC_PRIVATE
      11: 000246b4    20 OBJECT  GLOBAL DEFAULT   22 _r_debug@@GLIBC_2.4
      12: 00023f3c     4 OBJECT  GLOBAL DEFAULT   17 __libc_stack_end@@GLIBC_2.4
      13: 0001063c   160 FUNC    GLOBAL DEFAULT   10 _dl_tls_get_addr_soft@@GLIBC_PRIVATE
      14: 00014b50   308 FUNC    WEAK   DEFAULT   10 __libc_memalign@@GLIBC_2.4
      15: 00010920   192 FUNC    GLOBAL DEFAULT   10 _dl_deallocate_tls@@GLIBC_PRIVATE
      16: 00014d38    92 FUNC    WEAK   DEFAULT   10 calloc@@GLIBC_2.4
      17: 000245e4     4 OBJECT  GLOBAL DEFAULT   22 _dl_argv@@GLIBC_PRIVATE
      18: 0000f474  1384 FUNC    GLOBAL DEFAULT   10 _dl_mcount@@GLIBC_2.4
      19: 0001117c   204 FUNC    GLOBAL DEFAULT   10 _dl_tls_setup@@GLIBC_PRIVATE
      20: 0000e598     4 FUNC    GLOBAL DEFAULT   10 _dl_debug_state@@GLIBC_PRIVATE
      21: 00024060  1408 OBJECT  GLOBAL DEFAULT   20 _rtld_global@@GLIBC_PRIVATE
      22: 00010ce4   272 FUNC    GLOBAL DEFAULT   10 __tls_get_addr@@GLIBC_2.4
      23: 00011404   188 FUNC    GLOBAL DEFAULT   10 _dl_make_stack_executable@@GLIBC_PRIVATE
      24: 00014c84    12 FUNC    WEAK   DEFAULT   10 malloc@@GLIBC_2.4
      25: 000106dc   540 FUNC    GLOBAL DEFAULT   10 _dl_allocate_tls_init@@GLIBC_PRIVATE
      26: 00023db0   264 OBJECT  GLOBAL DEFAULT   17 _rtld_global_ro@@GLIBC_PRIVATE
      27: 00023f0c     4 OBJECT  GLOBAL DEFAULT   17 __stack_chk_guard@@GLIBC_2.4
      28: 00023f38     4 OBJECT  GLOBAL DEFAULT   17 __libc_enable_secure@@GLIBC_PRIVATE
      29: 00007bc0   456 FUNC    GLOBAL DEFAULT   10 _dl_rtld_di_serinfo@@GLIBC_PRIVATE
  
  Histogram for bucket list length (total of 17 buckets):
   Length  Number     % of total  Coverage
        0  2          ( 11.8%)
        1  6          ( 35.3%)     22.2%
        2  6          ( 35.3%)     66.7%
        3  3          ( 17.6%)    100.0%
  
  Histogram for `.gnu.hash' bucket list length (total of 17 buckets):
   Length  Number     % of total  Coverage
        0  2          ( 11.8%)
        1  8          ( 47.1%)     29.6%
        2  3          ( 17.6%)     51.9%
        3  3          ( 17.6%)     85.2%
        4  1          (  5.9%)    100.0%
  
  Version symbols section '.gnu.version' contains 30 entries:
   Addr: 0000000000000624  Offset: 0x000624  Link: 3 (.dynsym)
    000:   0 (*local*)       0 (*local*)       0 (*local*)       3 (GLIBC_PRIVATE)
    004:   3 (GLIBC_PRIVATE)   3 (GLIBC_PRIVATE)   2 (GLIBC_2.4)     3 (GLIBC_PRIVATE)
    008:   2 (GLIBC_2.4)     2 (GLIBC_2.4)     3 (GLIBC_PRIVATE)   2 (GLIBC_2.4)  
    00c:   2 (GLIBC_2.4)     3 (GLIBC_PRIVATE)   2 (GLIBC_2.4)     3 (GLIBC_PRIVATE)
    010:   2 (GLIBC_2.4)     3 (GLIBC_PRIVATE)   2 (GLIBC_2.4)     3 (GLIBC_PRIVATE)
    014:   3 (GLIBC_PRIVATE)   3 (GLIBC_PRIVATE)   2 (GLIBC_2.4)     3 (GLIBC_PRIVATE)
    018:   2 (GLIBC_2.4)     3 (GLIBC_PRIVATE)   3 (GLIBC_PRIVATE)   2 (GLIBC_2.4)  
    01c:   3 (GLIBC_PRIVATE)   3 (GLIBC_PRIVATE)
  
  Version definition section '.gnu.version_d' contains 3 entries:
    Addr: 0x0000000000000660  Offset: 0x000660  Link: 4 (.dynstr)
    000000: Rev: 1  Flags: BASE   Index: 1  Cnt: 1  Name: ld-linux.so.3
    0x001c: Rev: 1  Flags: none  Index: 2  Cnt: 1  Name: GLIBC_2.4
    0x0038: Rev: 1  Flags: none  Index: 3  Cnt: 2  Name: GLIBC_PRIVATE
    0x0054: Parent 1: GLIBC_2.4
  Attribute Section: aeabi
  File Attributes
    Tag_CPU_name: "5TE"
    Tag_CPU_arch: v5TE
    Tag_ARM_ISA_use: Yes
  </code></pre>
  
  <p>My next thought was that I know <code>glibc</code> provides an interface to the kernel, so perhaps it was expecting a kernel version different from <code>2.6.19.1</code>. However I am not sure how to determine what version of the kernel <code>ld</code> is targeting.</p>
  
  <p>I can post more information as its requested, open to any and all ideas. Thanks in advance. </p>