gdb error- "not in executable format: file format not recognized"

Question

When trying to debug (after compiling and linking) assembly 86-64x program called hello_world, I got a gdb error "not in executable format: file format not recognized".

ubuntu@ubuntu:~$ gdb hello_world
GNU gdb (Ubuntu 8.1-0ubuntu3) 8.1.0.20180409-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
"/home/ubuntu/hello_world": not in executable format: File format not recognized

I use Ubuntu 64x OS and gdb 8.1.0 64x.

I have looked for other answers, but couldn't understand what to do or the solution was for mac OS.

When running

`ubuntu@ubuntu:~$ file hello_world

I got

hello_world: ASCII text

After looking this answer I understood that gdb doesn't know what to do with this file, but I didn't figure out how to change the file's format.

My hello_world program:

global _start

section .text

 _start:
  mov rax,1
  mov rdi,1
  mov rsi,message
  mov rdx,13

  syscall

  mov rax,60
  xor rdi,rdi

  syscall

  section .data
  message: db "Hello, World",10

I have compiled and linked using the next commands:

   ubuntu@ubuntu:~$ nasm -felf64 hello_world
   ubuntu@ubuntu:~$ ld hello_world.o

What commands did you use to build this file, from what source? What do the first 10 lines look like? head hello_world. If it's really ASCII text, post it in your question so we can tell you what kind of file it actually is. Actual ELF executables are never mistaken for ASCII text- they start with a non-ASCII "magic number" in binary. — Peter Cordes
– Peter Cordes, Commented Mar 19, 2019 at 15:10
As Peter asked, how did you assemble and link? Are you loading the correct file (the output from the linker)? — Jester
– Jester, Commented Mar 19, 2019 at 15:16
So, hello_world is your source. You should really name it hello_world.asm as per convention. ld produces output named a.out by default so load that into gdb. You can specify other name with the -o option. Make sure not to overwrite your input though :) — Jester
– Jester, Commented Mar 19, 2019 at 15:25

fuz · Accepted Answer · 2019-03-19 17:16:58Z

7

   ubuntu@ubuntu:~$ nasm -felf64 hello_world
   ubuntu@ubuntu:~$ ld hello_world.o

hello_world is your source file; it's what you ran NASM on. Normally you'd name a NASM source file hello_world.asm, like a C hello_world.c.

The default output file for ld is a.out, so your command created an executable called a.out. If you want to create an executable called hello_world, you need to use
ld -o hello_world hello_world.o.

(Which would overwrite your source unless you first renamed it to .asm. This is why the convention is to use an extension on source files.)

You could have gotten a hint by running ls -lcrt to sort a directory listing by inode-change time. You'll see a.out at the bottom, after hello_world.o, which will remind you that ld created that instead of hello_world.

edited Mar 19, 2019 at 17:16

fuz

94.7k27 gold badges216 silver badges391 bronze badges

answered Mar 19, 2019 at 15:26

Peter Cordes

377k50 gold badges741 silver badges1k bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Peter Cordes Over a year ago

@fuz: GNU binutils ld accepts -o output before or after the input filenames. The man page only documents putting options before inputs, but in practice it works either way. I'll let your edit stand because it does match the documentation.

fuz Over a year ago

Not all toolchains work this way and it's not a good idea to popagate this bad habit. The Utility Syntax Guidelines of the POSIX standard specify that options come before operands; the GNU project is the major player not sticking to this convention.

Collectives™ on Stack Overflow

gdb error- "not in executable format: file format not recognized"

1 Answer 1

2 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

2 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related