Re: attempting to port to DFBSD

[Ori Bernstein <ori@xxxxxxxxxxxxxx>]

On Mon, 15 May 2017 12:38:13 -0400, Duck Deux <duck2@xxxxxxxxxxxxxx> wrote:

> hello.

Hi, 

> I am on DragonFly 4.8 https://www.dragonflybsd.org/
> 
> would like to port mc to this posixy system.

This is awesome!

> DFly forked from FBSD only 14 years ago, so I thought I could copy start-freebsd.s and abort-freebsd.s as start-dragonfly and abort-dragonfly.
> 
> I also copied over {ifreq, sys, syscall, syserrno}+freebsd-x64.myr, made new syscall table for sys+dragonfly. should probably do signals and errno too.

That's the right first step -- if the system call numbers are right, and the
structs that you're passing to the kernel are right, you're 80% of the way
there. Now comes debugging.

> currently mbld dies with sigsegv. checked with GDB, looks like the mmap in getmem() does not really get the memory.

So, here I'd write a simple test program that makes an allocation, and see
what kind of system calls it makes. Keep in mind that libstd has some
__init__ functions, so it may be possible that you don't even make it
to main.

So, for example, from the mc directory:

	$ echo <<EOF > hello.myr
	use std
	const main = {
		var sl : byte[:]
		std.put("Hi!\n")
		sl = std.slalloc(42)
	}
	EOF

	$ 6m hello.myr
	$ ld -o hello rt/_myrrt.o test.o -Llib/std -Llib/sys -lstd -lsys

Now, you've built a test binary by hand, and you can try running it. At the
end I'm going to put an annotated trace of what you see on my FreeBSD system:


> questions:
> - how should I print without std? syscall(Syswrite... ?

sys.write() should be equivalent to that.

For porting, though, I find the most useful tool is ktrace, dtrace, or any
other system call tracer. That allows you to figure out what system calls
you're *actually* doing, vs what system calls the OS thinks you're doing.

> - how to debug without gdb print command?

The print command works, although you have to peek at the assembly to figure
out *what* to print. It's a pain in the ass, and one of the biggest TODO items
I have to fix, once I move my code over to the QBE backend.

> - any common pitfalls when porting to new system? so I don't reinvent multiple wheels

Usually, it's pretty smooth sailing -- I did the bulk of netbsd in a night.
The biggest challenge is just auditing the structs in libsys and the system call
numbers, and it sounds like you've got that pretty close.

Once `sys` is ported, the only big challenge is the thread library, which involves
fiddly synchronization and some assembly to bring up and tear down the threads, and
has wildly varying primitives across platforms.

One thing that changed recently was that we started using .line directives
instead of dumping comments in assembly for source debugging in gdb, but sometimes
I found it useful to cross reference assembly which stiil had symbols. Let me know
if that would be useful for you, and I'd gladly bring it back behind a flag.

Given that there's not much written with threads right now, you can get
far enough without porting it. I'd accept a port without libthread, and
I'd be happy to work with you to finish that port.

> thx in advance.

No, thank you!

And, as promised, here's an example of what you see, and what exactly
is going on for each clump of syscalls. This will probably make it
easier to figure out where things are crashing and why.

	$ ktrace ./hello
	Hi!
	$ kdump

And the binary starts...

	21906 ktrace   RET   ktrace 0
	21906 ktrace   CALL  execve(0x7fffffffec5f,0x7fffffffe9e8,0x7fffffffe9f8)
	21906 ktrace   NAMI  "./hello"
	21906 hello    RET   execve 0

Sigpipe is stupid, so libsys turns it off by default. This may be a mistake
I'd consider undoing. This happens in lib/sys/setup+posixy.myr

	21906 hello    CALL  sigaction(SIGPIPE,0x7fffffffe7b8,0x7fffffffe798)
	21906 hello    RET   sigaction 0

Now we read /etc/hosts and such. We do a few allocations of different sizes to
hold the result; Each size class will allocate a new set of pages, so there's
a small flurry of them.

	21906 hello    CALL  mmap(0,0x800000,0x3<PROT_READ|PROT_WRITE>,0x1002<MAP_PRIVATE|MAP_ANON>,0xffffffffffffffff,0)
	21906 hello    RET   mmap 34366185472/0x800626000
	21906 hello    CALL  mmap(0,0x800000,0x3<PROT_READ|PROT_WRITE>,0x1002<MAP_PRIVATE|MAP_ANON>,0xffffffffffffffff,0)
	21906 hello    RET   mmap 34374574080/0x800e26000
	21906 hello    CALL  mmap(0,0x800000,0x3<PROT_READ|PROT_WRITE>,0x1002<MAP_PRIVATE|MAP_ANON>,0xffffffffffffffff,0)
	21906 hello    RET   mmap 34382962688/0x801626000
	21906 hello    CALL  mmap(0,0x800000,0x3<PROT_READ|PROT_WRITE>,0x1002<MAP_PRIVATE|MAP_ANON>,0xffffffffffffffff,0)
	21906 hello    RET   mmap 34391351296/0x801e26000
	21906 hello    CALL  mmap(0,0x800000,0x3<PROT_READ|PROT_WRITE>,0x1002<MAP_PRIVATE|MAP_ANON>,0xffffffffffffffff,0)
	21906 hello    RET   mmap 34399739904/0x802626000
	21906 hello    CALL  open(0x7fffffffe4bd,0<O_RDONLY>,<unused>0x1ff)
	21906 hello    NAMI  "/etc/hosts"
	21906 hello    RET   open 3
	21906 hello    CALL  mmap(0,0x800000,0x3<PROT_READ|PROT_WRITE>,0x1002<MAP_PRIVATE|MAP_ANON>,0xffffffffffffffff,0)
	21906 hello    RET   mmap 34408128512/0x802e26000
	21906 hello    CALL  read(0x3,0x803000040,0x1000)
	21906 hello    GIO   fd 3 read 1090 bytes
	<snip contents of my host file>
	21906 hello    RET   read 1090/0x442
	21906 hello    CALL  read(0x3,0x803000482,0xbbe)
	21906 hello    GIO   fd 3 read 0 bytes

The same happens with resolv.conf:

	21906 hello    RET   read 1090/0x442
	21906 hello    CALL  read(0x3,0x803000482,0xbbe)
	21906 hello    GIO   fd 3 read 0 bytes
	""
	21906 hello    RET   read 0
	21906 hello    CALL  close(0x3)
	21906 hello    RET   close 0
	21906 hello    CALL  mmap(0,0x800000,0x3<PROT_READ|PROT_WRITE>,0x1002<MAP_PRIVATE|MAP_ANON>,0xffffffffffffffff,0)
	21906 hello    RET   mmap 34416517120/0x803626000
	21906 hello    CALL  mmap(0,0x800000,0x3<PROT_READ|PROT_WRITE>,0x1002<MAP_PRIVATE|MAP_ANON>,0xffffffffffffffff,0)
	21906 hello    RET   mmap 34424905728/0x803e26000
	21906 hello    CALL  open(0x7fffffffe2f7,0<O_RDONLY>,<unused>0x1ff)
	21906 hello    NAMI  "/etc/resolv.conf"
	21906 hello    RET   open 3
	21906 hello    CALL  read(0x3,0x803002040,0x1000)
	21906 hello    GIO   fd 3 read 210 bytes
	21906 hello    RET   read 210/0xd2
	21906 hello    CALL  read(0x3,0x803002112,0xf2e)
	21906 hello    GIO   fd 3 read 0 bytes
	""
	21906 hello    RET   read 0
	21906 hello    CALL  close(0x3)
	21906 hello    RET   close 0

We then seed the RNG with the current time:

	21906 hello    CALL  clock_gettime(0,0x7fffffffe7a8)
	21906 hello    RET   clock_gettime 0
	21906 hello    CALL  clock_gettime(0,0x7fffffffe7a8)
	21906 hello    RET   clock_gettime 0

And now we run. There's not much we do.

	21906 hello    CALL  write(0x1,0x804000080,0x4)
	21906 hello    GIO   fd 1 wrote 4 bytes

We don't even mmap here, because we're allocating from the slabs that were
allocated during the init code.
-- 
    Ori Bernstein