Unix-like OS development - Shell. Conclusion (9)
It's time to write the first separate program for our kernel - the shell. It will be stored in a separate .elf file and launched by the init process when the kernel starts.
This is the last article in the development cycle of our operating system.
Build system (make, gcc, gas). Initial boot (multiboot). Launch (qemu). C library (strcpy, memcpy, strext).
C library (sprintf, strcpy, strcmp, strtok, va_list ...). Building the library in kernel mode and user application mode.
The kernel system log. Video memory Output to the terminal (kprintf, kpanic, kassert).
Dynamic memory, heap (kmalloc, kfree).
Organization of memory and interrupt handling (GDT, IDT, PIC, syscall). Exceptions
Virtual memory (page directory and page table).
Process. Scheduler. Multitasking. System calls (kill, exit, ps).
Character device drivers. System calls (ioctl, fopen, fread, fwrite). C library (fopen, fclose, fprintf, fscanf).
The file system of the kernel (initrd), elf, and its internals. System calls (exec). The shell as a complete program for the kernel.
User protection mode (ring3). Task state segment (tss).
In a previous article, we looked at character device drivers and wrote a terminal driver.
Now we have everything we need to create the first console application.
We will write the console application itself, which we will compile into a separate elf.
We will need to initialize the standard library and transfer control to the familiar main function.
Further in the loop, we simply read the line and execute the command.
Parsim commands through strtok_r if there are arguments.
In fact, we are just pulling system calls.
Let me remind you about the initialization of the standard library.
In the last lesson, we wrote the following function in the library:
It simply opens special files of the terminal driver for reading and writing, which corresponds to keyboard input and output to the screen.
After we have assembled our elf with a shell, it needs to be placed on the original kernel file system (initrd).
Initial ram disk is loaded by kernel loaders as a multiboot module, so we know the address in the memory of our initrd.
It remains to organize the file system for initrd, which is easy to do according to an article by James Molloy.
Therefore, the format will be as follows:
Next, remember the format of the 32 bit elf.
Here we are interested in the entry point and the address of the table of program headers.
The code and data section will be the first heading, and the stack section will be the second (according to the results of studying elf through objdump).
This info is enough to write an elf file loader.
We already know how to select pages for custom processes.
Therefore, we just need to select a sufficient number of pages for the headings and copy the contents into them.
We will write a function that will create a process based on the parsed elf file.
See how to parse elfik in the video tutorial.
We need to download only one program header with code and data, so we will not generalize and will focus on this case.
The most interesting thing here is the creation of a page directory and page table.
Pay attention, first we select the physical pages (mm_phys_alloc_pages), and then map them to the logical pages (mmu_occupy_user_page).
Here it is assumed that the pages in the physical memory are allocated continuously.
That's all. Now you can implement your own shell for your kernel! Watch the video tutorial and delve into the details.
I hope you found this series of articles useful.
We have not yet considered the protection rings with you, but due to the low relevance of the topic and the mixed reviews, we’ll continue to break.
I think you yourself are now ready for further research, for you and I have examined all the most important things.
Therefore, tighten the belt, and into battle! By writing your own operating system!
It took me about a month (if we consider full time for 6-8 hours a day) to implement everything that you and I learned from scratch.
Therefore, in 2-3 months you will be able to write a full-fledged OS with a real file system, which you and I did not manage to implement.
Just know that qemu does not know how to work with initrd of arbitrary format, and cuts it to 4kb, so you will need to either make it like in Linux, or use borsch instead of qemu.
If you know how to get around this problem, write in a personal letter, I will be very grateful to you.
That's all! Until the new no longer meet!
Watch the video tutorial for more information.
The source code in the git repository (you need the lesson9 branch).
1. James Molloy. Roll your own toy UNIX-clone OS.
2. Teeth. Assembler for DOS, Windows, Unix
3. Kalashnikov. Assembler is easy!
4. Tanenbaum. Operating Systems. Implementation and development.
5. Robert Love. Linux kernel Description of the development process.
This is the last article in the development cycle of our operating system.
Table of contents
Build system (make, gcc, gas). Initial boot (multiboot). Launch (qemu). C library (strcpy, memcpy, strext).
C library (sprintf, strcpy, strcmp, strtok, va_list ...). Building the library in kernel mode and user application mode.
The kernel system log. Video memory Output to the terminal (kprintf, kpanic, kassert).
Dynamic memory, heap (kmalloc, kfree).
Organization of memory and interrupt handling (GDT, IDT, PIC, syscall). Exceptions
Virtual memory (page directory and page table).
Process. Scheduler. Multitasking. System calls (kill, exit, ps).
Character device drivers. System calls (ioctl, fopen, fread, fwrite). C library (fopen, fclose, fprintf, fscanf).
The file system of the kernel (initrd), elf, and its internals. System calls (exec). The shell as a complete program for the kernel.
User protection mode (ring3). Task state segment (tss).
Shell as a complete kernel program
In a previous article, we looked at character device drivers and wrote a terminal driver.
Now we have everything we need to create the first console application.
We will write the console application itself, which we will compile into a separate elf.
/* * Elf entry point */ void start() { u_int errno; stdio_init(); errno = main(); stdio_deinit(); exit(errno); }
We will need to initialize the standard library and transfer control to the familiar main function.
int main() { char cmd[255]; while (1) { printf(prompt); flush(); scanf(cmd); if (!execute_command(cmd)) { break; } } return 0; }
Further in the loop, we simply read the line and execute the command.
Parsim commands through strtok_r if there are arguments.
static bool execute_command(char* cmd) { if (!strcmp(cmd, cmd_ps)) { /* show tasks list */ struct clist_definition_t *task_list; task_list = ps(); printf(" -- process list\n"); clist_for_each(task_list, print_task_info); } else if (!strcmp(cmd, cmd_clear)) { /* clear screen */ clear(); flush(); } else if (!strncmp(cmd, cmd_kill, strlen(cmd_kill))) { /* kill task */ char* save_ptr = null; strtok_r(cmd, " ", &save_ptr); char* str_tid = strtok_r(null, " ", &save_ptr); u_short tid = atou(str_tid); if (!kill(tid)) { printf(" There is no process with pid %u\n", tid); }; } else if (!strncmp(cmd, cmd_exit, strlen(cmd_exit))) { /* exit */ clear(); printf(prompt); flush(); return false; } else if (!strncmp(cmd, cmd_exec, strlen(cmd_exec))) { /* exec file on intrd */ char* save_ptr = null; strtok_r(cmd, " ", &save_ptr); char* str_file = strtok_r(null, " ", &save_ptr); exec(str_file); } else if (!strncmp(cmd, cmd_dev, strlen(cmd_dev))) { /* show device list */ struct clist_definition_t *dev_list; dev_list = devs(); printf(" -- device list\n"); clist_for_each(dev_list, print_dev_info); } else { printf(" There is no such command.\n Available command list:\n"); printf(" %s %s %s <pid> %s <file.elf> %s %s\n", cmd_ps, cmd_exit, cmd_kill, cmd_exec, cmd_clear, cmd_dev); } return true; }
In fact, we are just pulling system calls.
Let me remind you about the initialization of the standard library.
In the last lesson, we wrote the following function in the library:
extern void stdio_init() { stdin = fopen(tty_dev_name, MOD_R); stdout = fopen(tty_dev_name, MOD_W); asm_syscall(SYSCALL_IOCTL, stdout, IOCTL_INIT); asm_syscall(SYSCALL_IOCTL, stdin, IOCTL_READ_MODE_LINE); asm_syscall(SYSCALL_IOCTL, stdin, IOCTL_READ_MODE_ECHO); }
It simply opens special files of the terminal driver for reading and writing, which corresponds to keyboard input and output to the screen.
After we have assembled our elf with a shell, it needs to be placed on the original kernel file system (initrd).
Initial ram disk is loaded by kernel loaders as a multiboot module, so we know the address in the memory of our initrd.
It remains to organize the file system for initrd, which is easy to do according to an article by James Molloy.
Therefore, the format will be as follows:
extern struct initrd_node_t { unsigned char magic; /* magic number */ char name[8]; /* file name */ unsigned int offset; /* file base */ unsigned int length; /* file length */ }; extern struct initrd_fs_t { int count; /* files count */ struct initrd_node_t node[INITRD_MAX_FILES]; /* files headers */ };
Next, remember the format of the 32 bit elf.
struct elf_header_t { struct elf_header_ident_t e_ident; u16 e_type; u16 e_machine; u32 e_version; u32 e_entry; /* virtual address of entry point */ u32 e_phoff; /* program headers table offset */ u32 e_shoff; /* program headers sections table offset */ u32 e_flags; u16 e_ehsize; /* file header size */ u16 e_phentsize; /* single header size */ u16 e_phnum; /* headers count */ u16 e_shentsize; /* section header size */ u16 e_shnum; /* sections headers count */ u16 e_shstrndx; };
Here we are interested in the entry point and the address of the table of program headers.
The code and data section will be the first heading, and the stack section will be the second (according to the results of studying elf through objdump).
struct elf_program_header_t { u32 p_type; /* segment type */ u32 p_offset; /* segment offset from file begin */ u32 p_vaddr; /* target virtual address */ u32 p_paddr; /* target physical address */ u32 p_filesz; /* segment size in file */ u32 p_memsz; /* segment size in memory */ u32 p_flags; /* permissions and etc */ u32 p_align; /* alignment */ } attribute(packed);
This info is enough to write an elf file loader.
We already know how to select pages for custom processes.
Therefore, we just need to select a sufficient number of pages for the headings and copy the contents into them.
We will write a function that will create a process based on the parsed elf file.
See how to parse elfik in the video tutorial.
We need to download only one program header with code and data, so we will not generalize and will focus on this case.
/* * Api - execute elf as a task */ extern void elf_exec(struct elf_header_t* header) { assert(header->e_ident.ei_magic == EI_MAGIC); printf(MSG_KERNEL_ELF_LOADING, header->e_phnum); // elf_dump(header); size_t elf_base = (size_t)header; size_t entry_point = header->e_entry; struct task_mem_t task_mem; memset(&task_mem, 0, sizeof(struct task_mem_t)); // load sections in memory for (int i = 0; i < header->e_phnum; ++i) { struct elf_program_header_t* p_header = (void*)(header->e_phoff + elf_base + i * header->e_phentsize); task_mem.pages_count = (p_header->p_memsz / MM_PAGE_SIZE) + 1; if (p_header->p_memsz == 0) { continue; } // allocate pages assert(task_mem.pages_count > 0); assert(task_mem.pages == null); task_mem.pages = mm_phys_alloc_pages(task_mem.pages_count); void* section = (void*)(elf_base + p_header->p_offset); memcpy(task_mem.pages, section, p_header->p_memsz); // setup virtual memory task_mem.page_table = mmu_create_user_page_table(); task_mem.page_dir = mmu_create_user_page_directory(task_mem.page_table); for (int i = 0; i < task_mem.pages_count; ++i) { mmu_occupy_user_page(task_mem.page_table, (void*)((size_t)task_mem.pages + i * MM_PAGE_SIZE)); } } // create task u_short tid = next_tid++; assert(task_create(tid, (void*)entry_point, &task_mem)); // run task struct task_t* task; task = task_get_by_id(tid); task->status = TASK_RUNNING; strncpy(task->name, "elf", sizeof(task->name)); printf(MSG_KERNEL_ELF_LOADED); }
The most interesting thing here is the creation of a page directory and page table.
Pay attention, first we select the physical pages (mm_phys_alloc_pages), and then map them to the logical pages (mmu_occupy_user_page).
Here it is assumed that the pages in the physical memory are allocated continuously.
That's all. Now you can implement your own shell for your kernel! Watch the video tutorial and delve into the details.
Conclusion
I hope you found this series of articles useful.
We have not yet considered the protection rings with you, but due to the low relevance of the topic and the mixed reviews, we’ll continue to break.
I think you yourself are now ready for further research, for you and I have examined all the most important things.
Therefore, tighten the belt, and into battle! By writing your own operating system!
It took me about a month (if we consider full time for 6-8 hours a day) to implement everything that you and I learned from scratch.
Therefore, in 2-3 months you will be able to write a full-fledged OS with a real file system, which you and I did not manage to implement.
Just know that qemu does not know how to work with initrd of arbitrary format, and cuts it to 4kb, so you will need to either make it like in Linux, or use borsch instead of qemu.
If you know how to get around this problem, write in a personal letter, I will be very grateful to you.
That's all! Until the new no longer meet!
References
Watch the video tutorial for more information.
The source code in the git repository (you need the lesson9 branch).
Bibliography
1. James Molloy. Roll your own toy UNIX-clone OS.
2. Teeth. Assembler for DOS, Windows, Unix
3. Kalashnikov. Assembler is easy!
4. Tanenbaum. Operating Systems. Implementation and development.
5. Robert Love. Linux kernel Description of the development process.
All Articles