 |
blog.netbsd.org.atom.xml - sfeed_tests - sfeed tests and RSS and Atom files |
 |
git clone git://git.codemadness.org/sfeed_tests |
 |
Log |
|
Files |
|
Refs |
|
README |
|
LICENSE |
|
--- |
|
blog.netbsd.org.atom.xml (280500B) |
|
--- |
|
1 <?xml version="1.0" encoding='utf-8'?> |
|
2 <?xml-stylesheet type="text/xsl" href="https://blog.netbsd.org/roller-ui… |
|
3 <title type="html">NetBSD Blog</title> |
|
4 <subtitle type="html"></subtitle> |
|
5 <id>https://blog.netbsd.org/tnf/feed/entries/atom</id> |
|
6 <link rel="self" type="application/atom+xml" href="https://b… |
|
7 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
8 <updated>2022-03-09T23:57:54+00:00</updated> |
|
9 <generator uri="http://roller.apache.org" version="5.0.3 (1388864191… |
|
10 <entry> |
|
11 <id>https://blog.netbsd.org/tnf/entry/making_rockpro64_a_netbsd_… |
|
12 <title type="html">Making RockPro64 a NetBSD Server</title> |
|
13 <author><name>matthew green</name></author> |
|
14 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
15 <published>2022-03-09T23:57:54+00:00</published> |
|
16 <updated>2022-03-09T23:57:54+00:00</updated> |
|
17 <category term="/General" label="General" /> |
|
18 <category term="rockpro64" scheme="http://roller.apache.org/ns/t… |
|
19 <category term="aarch64" scheme="http://roller.apache.org/ns/tag… |
|
20 <category term="pine64" scheme="http://roller.apache.org/ns/tags… |
|
21 <category term="evbarm" scheme="http://roller.apache.org/ns/tags… |
|
22 <category term="aarch64eb" scheme="http://roller.apache.org/ns/t… |
|
23 <content type="html"><p>The time has come to upgrade my Su… |
|
24 |
|
25 <p>After much searching, I've decided on <a href="htt… |
|
26 |
|
27 <p>In my SATA setup, I have 3 SSDs with a JMicron 585 card in the … |
|
28 |
|
29 |
|
30 <p>In the last year or so I've been working on making NetBSD/… |
|
31 <ul><li>other-endian access disklabels, FFS file-systems in … |
|
32 <li>the "look 64 sectors later" for RAIDFrame partitions… |
|
33 <li>other-endian access to RAIDFrame labels in the kernel</li&g… |
|
34 <li>updated the U-Boot package and featureset to include AHCI/SATA… |
|
35 </ul></p> |
|
36 <p>There are not too many special actions needed for this sort of … |
|
37 |
|
38 <p>To install U-Boot into the SD/eMMC card (can run on any system,… |
|
39 <pre> |
|
40 # dd if=/usr/pkg/share/u-boot/rockpro64/rksd_loader.img seek=64 of=/dev/… |
|
41 </pre> |
|
42 <p> |
|
43 To install U-Boot into the SPI flash (must be run on the host, and lives… |
|
44 <pre> |
|
45 # dd if=/usr/pkg/share/u-boot/rockpro64/rkspi_loader.img bs=64k of=/dev/… |
|
46 </pre> |
|
47 </p> |
|
48 |
|
49 <p>When booting from NVMe or SATA, one must drop to the U-Boot pro… |
|
50 |
|
51 <p>The original value for me: |
|
52 <pre> |
|
53 => printenv boot_targets |
|
54 boot_targets=mmc1 usb0 mmc0 nvme0 scsi0 pxe dhcp sf0 |
|
55 </pre> |
|
56 </p> |
|
57 |
|
58 <p>Which is then adjusted and saved with eg: |
|
59 <pre> |
|
60 => setenv boot_targets nvme0 scsi0 mmc1 usb0 mmc0 pxe dhcp sf0 |
|
61 => saveenv |
|
62 Saving Environment to SPIFlash... Erasing SPI flash...Writing to SPI fla… |
|
63 OK |
|
64 </pre></p> |
|
65 |
|
66 <p>(In this list, mmc1 is the SD slot, mmc0 is the eMMC card, pxe … |
|
67 |
|
68 <p>There are some minor issues with the RockPro64. It has no abil… |
|
69 </content> |
|
70 </entry> |
|
71 <entry> |
|
72 <id>https://blog.netbsd.org/tnf/entry/project_report_add_support… |
|
73 <title type="html">Project Report: Add support for chdir(2) supp… |
|
74 <author><name>martin</name></author> |
|
75 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
76 <published>2021-11-22T18:01:54+00:00</published> |
|
77 <updated>2021-11-23T16:38:20+00:00</updated> |
|
78 <category term="/Development" label="Development" /> |
|
79 <category term="projects" scheme="http://roller.apache.org/ns/ta… |
|
80 <category term="tnf" scheme="http://roller.apache.org/ns/tags/" … |
|
81 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/"… |
|
82 <summary type="html"><p>Piyush Sachdeva finished the "… |
|
83 His code is already in -current and will be part of NetBSD 10.</p> |
|
84 <p>Originally submitted as a proposal for GSoC, but unfortunately… |
|
85 <p>The NetBSD Foundation decided to nevertheless run the project a… |
|
86 <content type="html"><h3>This post was written by Piyush S… |
|
87 |
|
88 <h2>Abstract</h2> |
|
89 |
|
90 <p>The primary goal of the project was to extend posix_spawn(3) to… |
|
91 for the newly created child process. Two functions were supposed to be i… |
|
92 namely posix_spawn_file_actions_addchdir() and |
|
93 posix_spawn_file_actions_addfchdir(), to support both chdir(2) and |
|
94 fchdir(2) respectively. |
|
95 posix_spawn() is a POSIX standard method |
|
96 responsible for creating and executing new child processes.</p> |
|
97 |
|
98 <h2>Implementation</h2> |
|
99 <p>The original code can be found at <a href="https://gith… |
|
100 <p>The implementation plan was discussed and made with the guidanc… |
|
101 mentors Martin Husemann and Joerg Sonnenberger. The plan was divided into |
|
102 three phases each corresponding to the specific part of The NetBSD code-… |
|
103 supposed to be touched:</p> |
|
104 |
|
105 <h3>User-Land</h3> |
|
106 <p>The following actions were performed in the user-land to set th… |
|
107 kernel-space. |
|
108 <ul> |
|
109 <li>Add another member to the posix_spawn_file_actions_t s… |
|
110 which would hold the path to chdir to.</li> |
|
111 </li> |
|
112 <li>Implement the two functions posix_spawn_file_actions_a… |
|
113 and posix_spawn_file_actions_addfchdir(). These … |
|
114 <ol> |
|
115 <li>allocate memory for another posix_spawn_file… |
|
116 posix_spawn_file_actions_t array.</li> |
|
117 <li>take the path/file descriptor from the user … |
|
118 relative field of the newly allocated fi… |
|
119 </ol> |
|
120 </li> |
|
121 <li>The final step was to add the prototypes for the two n… |
|
122 `src/include/spawn.h' header file |
|
123 </li> |
|
124 </ul> |
|
125 |
|
126 <p> |
|
127 Once the aforementioned changes were made, the only thing left to do was… |
|
128 kernel support these two new functions.</p> |
|
129 |
|
130 <h3>Kernel-Space</h3> |
|
131 <p>The following actions were performed inside the kernel space.&l… |
|
132 <ul> |
|
133 <li>The three functions in the `src/sys/kern_exec.c' … |
|
134 posix_spawn_file_actions were edited: |
|
135 <ul> |
|
136 <li>posix_spawn_fa_alloc() was adjusted to make su… |
|
137 posix_spawn_file_actions_addchdir() gets copied … |
|
138 to the kernel-space. |
|
139 </li> |
|
140 <li>Similarly posix_spawn_fa_free() was adjusted t… |
|
141 memory allocated in case of FAE_CHDIR gets freed… |
|
142 </li> |
|
143 <li>Finally, two new cases FAE_CHDIR &amp; FAE… |
|
144 handle_posix_spawn_file_actions(). In each of th… |
|
145 a call to one of the two newly created functions… |
|
146 do_sys_chdir() and do_sys_fchdir() was made resp… |
|
147 </li> |
|
148 </ul> |
|
149 |
|
150 <p>Note: At the time of code integration, a helper functio… |
|
151 Christos Zoulas. This function aimed to reduce t… |
|
152 in both posix_spawn_fa__free() and posix_spawn_f… |
|
153 </li> |
|
154 |
|
155 <li>Two new functions, similar to the already present sys_… |
|
156 `src/sys/vfs_syscalls.c' were created. Namely do_sy… |
|
157 were written with two specific thoughts in mind: |
|
158 <ul> |
|
159 <li>By default sys_chdir() and sys_fchdir() took s… |
|
160 The purpose of the new functions was to replace … |
|
161 const char * and an int type parameter respectiv… |
|
162 <li>The do_sys_chdir() also replaced UIO_USERSPACE… |
|
163 UIO_SYSSPACE. This was done because the chdir pa… |
|
164 function already resided in the Kernel-space due to the … |
|
165 posix_spawn_fa_alloc().</li> |
|
166 </ul></li> |
|
167 <li>Finally, the prototypes for the newly written function… |
|
168 `src/sys/sys/vfs_syscalls.h' file and this file was… |
|
169 'sys/kern/kern_exec.c'.</li> |
|
170 |
|
171 </ul> |
|
172 |
|
173 <p>Note: Similar to the above changes of user-land and kernel-spac… |
|
174 `src/sys/compat/netbsd/netbsd32.h' and `netbsd32_execve.c'. Th… |
|
175 deal with the new file actions member. However, these changes were made … |
|
176 by Martin Husemann.</p> |
|
177 |
|
178 |
|
179 <p>With most of addition of new features being done, all that rema… |
|
180 |
|
181 |
|
182 <h3>Testing &amp; Documentation</h3> |
|
183 <ul> |
|
184 <li>A total of ten new test cases have been added to the |
|
185 `src/tests/lib/libc/gen/posix_spawn/t_spawn.c' file… |
|
186 <li>Three utility functions were also used to aid in testi… |
|
187 one new function was written and two existing functions … |
|
188 and empty_outfile()) from `t_fileactions.c' were us… |
|
189 To make sure that the 2 existing functions were shared b… |
|
190 and `t_fileactions.c' a new header and C file was c… |
|
191 `fa_spawn_utils.c'. |
|
192 Following this, the bodies of both the functions were mo… |
|
193 `t_fileactions.c' to `fa_spawn_utils.c' and th… |
|
194 corresponding header file.</li> |
|
195 <li>The general approach that was taken to all test cases … |
|
196 posix_spawn() execute ``/bin/pwd'' and write t… |
|
197 Then read the file and do string comparison. The third f… |
|
198 was written for just this purpose.</li> |
|
199 <li>The ten test cases cover the following scenarios: |
|
200 <ul> |
|
201 <li>Absolute path test - for both chdir and fchdir… |
|
202 <li>Relative path test - for both chdir and fchdir… |
|
203 <li>Trying to open a file instead of directory - f… |
|
204 <li>Invalid path/file descriptor (fd=-1) - for bot… |
|
205 <li>Trying to open a directory without access perm… |
|
206 <li>Opening a closed file descriptor for fchdir.&l… |
|
207 </ul></li> |
|
208 <li>The first 8 test cases had a lot of repetitive code. T… |
|
209 another function was created i.e spawn_chdir(). |
|
210 This function included a huge chunk of the common code a… |
|
211 for those first 8 test cases.</li> |
|
212 </ul> |
|
213 |
|
214 <h4>Documentation:</h4> |
|
215 <p>In this matter, a complete man page is written which explains b… |
|
216 posix_spawn_file_actions_addchdir() and posix_spawn_file_actions_addfchd… |
|
217 The content of the manual page is taken from the POSIX documentation pro… |
|
218 </p> |
|
219 |
|
220 |
|
221 <h2>Issues</h2> |
|
222 <p>Since the project was well planned from the beginning, it resul… |
|
223 <ul> |
|
224 <li>The user-land was the most straight forward part of th… |
|
225 trouble sailing through it.</li> |
|
226 <li>Kernel space was where things got a bit complicated, a… |
|
227 to pre-existing functions.</li> |
|
228 <li>I was completely new to using atf(7) and groff(1). The… |
|
229 to understand the respective man pages and become comfor… |
|
230 documentation part.</li> |
|
231 </ul> |
|
232 |
|
233 <p> |
|
234 Most of the issues faced were generally logistical. As it was my first t… |
|
235 kernel project, I was new to building from source, Virtual Machines and … |
|
236 But luckily, I had great help from my mentors and the entire NetBSD comm… |
|
237 |
|
238 <h2>Thanks</h2> |
|
239 <p>I would like to express my heartfelt gratitude to The NetBSD Fo… |
|
240 giving me this opportunity and sponsoring the Project. |
|
241 This project would not have been possible without the constant support a… |
|
242 encouragement of both my mentors Martin Husemann and Joerg Sonnenberger. |
|
243 My gratitude to Christos Zoulas who worked on the crucial part of integr… |
|
244 A special mention to all of the other esteemed NetBSD developers, |
|
245 who have helped me navigate through the thick and thin of this project a… |
|
246 have answered even my most trivial questions.</p> |
|
247 </content> |
|
248 </entry> |
|
249 <entry> |
|
250 <id>https://blog.netbsd.org/tnf/entry/wifi_project_status_update… |
|
251 <title type="html">wifi project status update</title> |
|
252 <author><name>martin</name></author> |
|
253 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
254 <published>2021-08-26T17:46:53+00:00</published> |
|
255 <updated>2021-08-26T17:46:53+00:00</updated> |
|
256 <category term="/Development" label="Development" /> |
|
257 <category term="funded" scheme="http://roller.apache.org/ns/tags… |
|
258 <category term="wifi" scheme="http://roller.apache.org/ns/tags/"… |
|
259 <category term="freebsd" scheme="http://roller.apache.org/ns/tag… |
|
260 <summary type="html"><p>About a year ago the <a href=&q… |
|
261 <p>Here is where we are today, what needs to be done and how thing… |
|
262 <content type="html"><p>After initial work on the <a hr… |
|
263 |
|
264 <p>Most of the slow down was due to me not being available for thi… |
|
265 |
|
266 <p>However, there were other obstacles and unexpected issues on th… |
|
267 <ul> |
|
268 <li>bpf taps are handled differently in the new stack and some s… |
|
269 <li>Many wifi drivers seem to be in a, let's say, slightly f… |
|
270 This is not a happy base to start converting drivers from.</li> |
|
271 <li>After the great success of usbnet(9) for USB ethernet drivers… |
|
272 <li>I have been hitting a bug in the urtwn(4) driver used for ini… |
|
273 </ul> |
|
274 |
|
275 <p>The current state of driver conversion and what drivers are sti… |
|
276 |
|
277 <p>Next steps ahead are:</p> |
|
278 <ul> |
|
279 <li><s>make another pass over documentation and improve thi… |
|
280 <li>sync the branch with HEAD and keep tracking it more closely&l… |
|
281 <li>convert run(4) to usbwifi</li> |
|
282 <li>revisit rtwn(4) and decide if/how it should be merged with ur… |
|
283 <li>revisit iwm(4) and make it work fully</li> |
|
284 <li>convert all other drivers, starting with the ones I have hard… |
|
285 </ul> |
|
286 |
|
287 <p>Currently it is not clear if this branch can be merged to HEAD … |
|
288 </content> |
|
289 </entry> |
|
290 <entry> |
|
291 <id>https://blog.netbsd.org/tnf/entry/support_for_chdir_2_in</id> |
|
292 <title type="html">Support for chdir(2) in posix_spawn(3)</title> |
|
293 <author><name>martin</name></author> |
|
294 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
295 <published>2021-06-10T15:28:05+00:00</published> |
|
296 <updated>2021-06-10T15:28:05+00:00</updated> |
|
297 <category term="/Development" label="Development" /> |
|
298 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/"… |
|
299 <category term="posix_spawn" scheme="http://roller.apache.org/ns… |
|
300 <summary type="html"><p>Piyush Sachdeva is working on an e… |
|
301 |
|
302 <p>He applied as a GSoC student, but unfortunately we only got a s… |
|
303 offered Piyush to work on it by TNF funding outside of the official GSo… |
|
304 |
|
305 <p>In this post Piyush introduces himself and the project. He alre… |
|
306 <content type="html"><h3>This post was written by Piyush S… |
|
307 |
|
308 <p>What really happens when you double click an icon on your deskt… |
|
309 |
|
310 <h1>Support for chdir(2) in posix_spawn(3)</h1> |
|
311 |
|
312 <p>Processes are the bread and butter of your operating system. Th… |
|
313 you double click an icon, that particular program gets loaded in your |
|
314 Random Access Memory (RAM) and your operating system starts to run it. At |
|
315 this moment the program becomes a process. Though you can only see the e… |
|
316 of your process, the operating system (the Kernel) is always running a l… |
|
317 of processes in the background to facilitate you.</p> |
|
318 |
|
319 <p>From the moment you hit that power button, everything that happ… |
|
320 screen is the result of some or the other process. In this post we are |
|
321 going to talk about one such interface which helps in creation of your |
|
322 programs.</p> |
|
323 |
|
324 <h2>The fork() & exec() shenanigans</h2> |
|
325 |
|
326 <p>The moment a computer system comes alive, it launches a bunch of |
|
327 processes. For the purpose of this blog let&#x2019s call them, &… |
|
328 processes&#x2019. These processes run in perpetuity, provided the co… |
|
329 switched on. One such process is <a href="#note1">init/s… |
|
330 This &#x2018init&#x2019 master process owns all the other proces… |
|
331 either directly or indirectly.</p> |
|
332 |
|
333 <p>Operating systems are elegant, majestic software that work |
|
334 seamlessly under the hood. They do so much without even breaking a sweat |
|
335 (unless it&#x2019s Windows). Let's consider a scenario where yo… |
|
336 take a trip down memory lane and burst open those old photos. The &#… |
|
337 master process&#x2019 just can&#x2019t terminate itself and let … |
|
338 photos. What if you unknowingly open a malicious file, which corrupts all |
|
339 your data? So init doesn&#x2019t just exit, rather it employs fork()… |
|
340 to start a new process. The fork() function is used to create child |
|
341 processes which are an exact copy of their parents. Whichever process ca… |
|
342 fork, gets duplicated. The newly created process becomes the child of the |
|
343 original running process and the original running process is called the |
|
344 parent. Just how parents look after their kids, the parent process makes |
|
345 sure that the child process doesn't do any mischief. So now you hav… |
|
346 exactly similar processes running in your computer.</p> |
|
347 |
|
348 <img id="flowchar", src="//www.NetBSD.org/~martin/flow… |
|
349 |
|
350 <p>One might think that the newly created child process doesn&… |
|
351 us. But actually, it does. Now exec() comes into the picture. What exec() |
|
352 does is, it replaces any process which calls it. So what if we replace t… |
|
353 process, the one we just thought to be useless, with our photos? That&ap… |
|
354 exactly what we are going to do indeed. This will result in replacement … |
|
355 the fork() created child process with your photos. Therefore, the master |
|
356 init process is still running and you can also enjoy your photos with no |
|
357 threat to your data.</p> |
|
358 |
|
359 |
|
360 <cite> |
|
361 &#x201cNeither abstraction nor simplicity is a substitute for gettin… |
|
362 Sometimes, you just have to do the right thing, and when you do, it is w… |
|
363 better than the alternatives. There are lots of ways to design APIs for |
|
364 process creation; however, the combination of fork() and exec() is simple |
|
365 and immensely powerful. Here, the UNIX designers simply got it right.&am… |
|
366 </cite> <a href="#note2">Lampson&#x2019s Law -… |
|
367 |
|
368 <p>Now you could ask me, `But what about the title, some &#x20… |
|
369 thing?´ Don&#x2019t worry, that&#x2019s next.</p> |
|
370 |
|
371 <h2>posix_spawn()</h2> |
|
372 |
|
373 <p>posix_spawn() is an alternative to the fork() + exec() routine.… |
|
374 implements fork() and exec(), but not directly (as that would make it |
|
375 slow, and we all need everything to be lightning fast). What actually |
|
376 happens is that posix_spawn() only implements the functionality of the |
|
377 fork() + exec() routines, but in one single call. However, because fork(… |
|
378 exec() is a combination of two different calls, there is a lot of room f… |
|
379 customization. Whatever software you are running on your computer, calls |
|
380 these routines on its own and does the necessary. Meanwhile a lot is |
|
381 cooking in the background. Between the call to fork() and exec() there is |
|
382 plenty of leeway for tweaking different aspects of the exec-ing process. |
|
383 But posix_spawn doesn&#x2019t bear this flexibility and therefore ha… |
|
384 limitations. It does take a lot of parameters to give the caller some |
|
385 flexibility, but it is not enough.</p> |
|
386 |
|
387 <p>Now the question before us is, |
|
388 &#x201cIf fork() + exec() is so much more powerful, then why have, |
|
389 or use the posix_spawn() routine?&#x201d The answer to that is, that |
|
390 <a href="#note3">fork() and exec()</a> are UNIX sy… |
|
391 They are not present in operating systems that are not a derivative of U… |
|
392 Eg- Windows implements a family of spawn functions.<br/> |
|
393 There is another issue with fork() (not exec() ), which in reality is one |
|
394 of the biggest reasons behind the growth of posix_spawn(). The outline of |
|
395 the issue is that, creating child processes in multi-threaded programs i… |
|
396 whole another ball game altogether.</p> |
|
397 |
|
398 |
|
399 <p>Concurrency is one of those disciplines in operating systems wh… |
|
400 order in which the cards are going to unravel is not always how you expe… |
|
401 them to. Multi-threading in a program is a way to do different and indep… |
|
402 program simultaneously, to save time. No matter how jazzy or intelligent… |
|
403 multi-threaded programs require an eagle&#x2019s eye as they can oft… |
|
404 of holes. Though the &#x201ctasks&#x201d are different and indep… |
|
405 share a few common attributes. When these different tasks due to |
|
406 concurrency start running in parallel, a data race begins to access those |
|
407 shared attributes. To not wreak havoc, there are mechanisms through whic… |
|
408 when modifying/accessing these common attributes (Critical Section) we c… |
|
409 provide a sort of mutual exclusion (locks/conditional variables) - only |
|
410 letting one of the processes modify the shared attribute at a time. Here |
|
411 when things are already so intricate due to multithreading, and to top it |
|
412 off, we start creating child processes. Complications are bound to arise. |
|
413 When one of the threads from the multi-threaded program calls fork() to |
|
414 create a child process, fork() does clone everything (variables, their |
|
415 states, functions, etc) but it fails to clone other threads (though this… |
|
416 required at all times).</p> |
|
417 |
|
418 <p>The child process now knows only about that one thread which ca… |
|
419 But all the other attributes of the child that were inherited from |
|
420 the parent (locks, mutexes) are set from the parent&#x2019s address … |
|
421 (considering multiple threads). So there is no way for the child process… |
|
422 know which attributes conform to which parts of the parent. Also, those |
|
423 mechanisms that we used to provide mutual exclusion, like locks and |
|
424 conditional variables, need to be reset. This reset step is essential |
|
425 in letting the parent access it&#x2019s attributes. Failing this res… |
|
426 To put it simply, you can see how difficult things |
|
427 have become all of a sudden. The posix_spawn() call is free from these |
|
428 limitations of fork() encountered in multi-threaded programs. However, as |
|
429 mentioned by me earlier, there needs to be enough rope to meet all the |
|
430 requirements before posix_spawn() does the implicit exec().</p> |
|
431 |
|
432 |
|
433 <h2>About my Project</h2> |
|
434 |
|
435 <p>Hi, I am Piyush Sachdeva and I am going to start a project whic… |
|
436 on relaxing one limitation of posix_spawn - changing the current directo… |
|
437 of the child process, before the said call to exec() is made. This is not |
|
438 going to restrict it to the parent&#x2019s current working directory… |
|
439 passing the new directory as one of the parameters will do the trick. |
|
440 Resolving all the impediments would definitely be marvelous. Alas! That … |
|
441 not possible. Every attempt to resolve even a single hindrance can create |
|
442 plenty of new challenges.</p> |
|
443 |
|
444 <p>As already mentioned by me, posix_spawn() is a POSIX standard. … |
|
445 effect of my project will probably be reflected in the next POSIX releas… |
|
446 I came across this project through Google Summer of Code 2021. It was be… |
|
447 offered by The NetBSD Foundation Inc. However, as the slots for |
|
448 Google Summer of Code were numbered, my project didn&#x2019t make th… |
|
449 Nevertheless, the Core Team at The NetBSD Foundation offered me to work … |
|
450 the project and even extended a handsome stipend. I will forever be |
|
451 grateful to The NetBSD Foundation for this opportunity.</p> |
|
452 |
|
453 <h2>Notes</h2> |
|
454 |
|
455 <ul> |
|
456 <li>init, systemd & launchd are system daemon processes. init … |
|
457 historical process present since System III UNIX. systemd was the replac… |
|
458 for the authentic init which was written for the Linux kernel. |
|
459 launchd is the MacOS alternative of init/systemd.</li> |
|
460 |
|
461 <li>This is taken from Operating Systems: The Three Easy Pieces bo… |
|
462 Andrea C. Arpaci-Dusseau and Remzi H. Arpaci-Dusseau.</li> |
|
463 |
|
464 <li>UNIX is the original AT&T UNIX operating system developed … |
|
465 Labs research center, headed by Ken Thompson and Dennis Ritchie.</li&… |
|
466 </ul> |
|
467 |
|
468 <h2>References</h2> |
|
469 |
|
470 <ol> |
|
471 <li> <a name="note1"> Operating Systems: Three… |
|
472 Remzi H. Arpaci-Dusseau.</a></li> |
|
473 <li> <a name="note2"> Advanced Programming in the… |
|
474 and Stephen A. Rago.</a></li> |
|
475 <li> <a name="note3">UNIX and Linux System Admi… |
|
476 Synder, Trent R. Hein, Ben Whaley and Dan Mackin.</a></li> |
|
477 </ol></content> |
|
478 </entry> |
|
479 <entry> |
|
480 <id>https://blog.netbsd.org/tnf/entry/public_netbsd_irc_channels… |
|
481 <title type="html">Public NetBSD IRC chat channels moved to Libe… |
|
482 <author><name>Nia Alarie</name></author> |
|
483 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
484 <published>2021-05-30T18:23:29+00:00</published> |
|
485 <updated>2021-05-30T18:24:49+00:00</updated> |
|
486 <category term="/General" label="General" /> |
|
487 <summary type="html"><pHi everyone,</p> |
|
488 |
|
489 <p>Due to the unfortunate situation regarding changes in administr… |
|
490 freenode.net, and the resulting chaos, we have decided to move the public |
|
491 NetBSD IRC channels from freenode to irc.libera.chat.</p> |
|
492 |
|
493 <p>This includes:</p> |
|
494 |
|
495 <ul> |
|
496 <li><a href="https://web.libera.chat/#NetBSD">#Net… |
|
497 <li><a href="https://web.libera.chat/#netbsd-code">… |
|
498 <li><a href="https://web.libera.chat/#pkgsrc">#pkg… |
|
499 </ul> |
|
500 |
|
501 <p>You can find information on connecting to Libera at <a href=… |
|
502 <content type="html"><p>Hi everyone,</p> |
|
503 |
|
504 <p>Due to the unfortunate situation regarding changes in administr… |
|
505 freenode.net, and the resulting chaos, we have decided to move the public |
|
506 NetBSD IRC chat channels from freenode to irc.libera.chat.</p> |
|
507 |
|
508 <p>This includes:</p> |
|
509 |
|
510 <ul> |
|
511 <li><a href="https://web.libera.chat/#NetBSD">#Net… |
|
512 <li><a href="https://web.libera.chat/#netbsd-code">… |
|
513 <li><a href="https://web.libera.chat/#pkgsrc">#pkg… |
|
514 </ul> |
|
515 |
|
516 <p>You can find information on connecting to Libera at <a href=… |
|
517 </entry> |
|
518 <entry> |
|
519 <id>https://blog.netbsd.org/tnf/entry/netbsd_9_2_released</id> |
|
520 <title type="html">NetBSD 9.2 released</title> |
|
521 <author><name>Nia Alarie</name></author> |
|
522 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
523 <published>2021-05-17T14:06:03+00:00</published> |
|
524 <updated>2021-05-17T14:37:04+00:00</updated> |
|
525 <category term="/Release engineering" label="Release engineering… |
|
526 <summary type="html"><p> The NetBSD Project is pleased to … |
|
527 |
|
528 <p>As well as the usual bug, stability, and security fixes, this r… |
|
529 |
|
530 <p><a href="//www.NetBSD.org/releases/formal-9/NetBSD-9.2.… |
|
531 <content type="html"><p> The NetBSD Project is pleased to … |
|
532 |
|
533 <p>As well as the usual bug, stability, and security fixes, this r… |
|
534 |
|
535 <p><a href="//www.NetBSD.org/releases/formal-9/NetBSD-9.2.… |
|
536 </entry> |
|
537 <entry> |
|
538 <id>https://blog.netbsd.org/tnf/entry/aiomixer_x_open_curses_and… |
|
539 <title type="html">aiomixer, X/Open Curses and ncurses, and othe… |
|
540 <author><name>Nia Alarie</name></author> |
|
541 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
542 <published>2021-05-12T13:35:14+00:00</published> |
|
543 <updated>2021-05-12T14:48:40+00:00</updated> |
|
544 <category term="/Development" label="Development" /> |
|
545 <summary type="html"><p>aiomixer, X/Open Curses and ncurse… |
|
546 <content type="html"><p> |
|
547 aiomixer is an application that I've been maintaining outside of Ne… |
|
548 few years. It was available as a package, and was a "graphical"… |
|
549 terminal-based) mixer for NetBSD's audio API, inspired by programs … |
|
550 For some time I've thought that it should be integrated into the Ne… |
|
551 base system - it's small and simple, very useful, and many develope… |
|
552 and users had it installed (some told me that they would install it on a… |
|
553 of their machines that needed audio output). |
|
554 For my particular use case, as well as my NetBSD laptop, I have some sma… |
|
555 NetBSD machines around the house plugged into speakers that I play music… |
|
556 Sometimes I like to SSH into them to adjust the playback volume, and it&… |
|
557 easier to do visually than with |
|
558 <a href="https://man.NetBSD.org/mixerctl.1">mixerctl(1)&… |
|
559 </p> |
|
560 |
|
561 <p> |
|
562 However, there was one problem: when I first wrote aiomixer 2 years ago, |
|
563 I was intimidated by the curses API, so opted to use the |
|
564 <a href="https://invisible-island.net/cdk/">Curses Devel… |
|
565 instead. |
|
566 This turned out to be a mistake, as not only was CDK inflexible for an |
|
567 application like aiomixer, it introduced a hard dependency on ncurses. |
|
568 </p> |
|
569 |
|
570 <h2>X/Open Curses and ncurses</h2> |
|
571 |
|
572 <p> |
|
573 Many people think ncurses is the canonical way to develop terminal-based |
|
574 applications for Unix, but it's actually an implementation of the |
|
575 <a href="https://pubs.opengroup.org/onlinepubs/7908799/xcurses/c… |
|
576 There's a few other Curses implementations: |
|
577 </p> |
|
578 |
|
579 <ul> |
|
580 <li><a href="https://man.netbsd.org/curses.3"… |
|
581 <li><a href="https://docs.oracle.com/cd/E36784_01/… |
|
582 <li><a href="https://en.wikipedia.org/wiki/PDCurse… |
|
583 </ul> |
|
584 |
|
585 <p> |
|
586 NetBSD curses is descended from the original BSD curses, but contains |
|
587 many useful extensions from ncurses as well. We use it all over the |
|
588 base system, and for most packages in pkgsrc. |
|
589 It's also been |
|
590 <a href="https://github.com/sabotage-linux/netbsd-curses"&g… |
|
591 including Linux. |
|
592 As far as I'm aware, NetBSD is one of the last operating systems le… |
|
593 that doesn't primarily depend on ncurses. |
|
594 </p> |
|
595 |
|
596 <p> |
|
597 There's one crucial incompatibility, however: ncurses exposes its |
|
598 internal data structures, NetBSD libcurses keeps them opaque. |
|
599 Since CDK development is very tied to ncurses development (they have |
|
600 the same maintainer), CDK peeks into those structures, and can't |
|
601 be used with NetBSD libcurses. |
|
602 There are also a few places where ncurses breaks with X/Open Curses, |
|
603 like |
|
604 <a href="https://github.com/irssi/irssi/pull/1305">this … |
|
605 </p> |
|
606 |
|
607 <h2>Rewriting aiomixer</h2> |
|
608 |
|
609 <p> |
|
610 I was able to rewrite aiomixer in a few days using only my free time |
|
611 and NetBSD libcurses. It's now been imported to the base system. |
|
612 It was a good lesson in why Curses isn't actually that intimidating… |
|
613 while there are many functions, they're mostly variations on the |
|
614 same thing. Using Curses directly resulted in a much lighter and |
|
615 more usable application, and provided a much better fit for the |
|
616 types of widgets I needed. |
|
617 </p> |
|
618 |
|
619 <p> |
|
620 Many people also provided testing, and I learned a lot about |
|
621 how different terminal attributes should be used in the process. |
|
622 NetBSD is probably one of the few communities where you'll get |
|
623 easy and direct feedback on how to not only make your software |
|
624 work well in a variety of terminal emulators, but also old school |
|
625 hardware terminals. During development, I was also able to find |
|
626 a strange bug in the curses library's window resizing function. |
|
627 </p> |
|
628 |
|
629 <p> |
|
630 The API support was also improved, and the new version of aiomixer |
|
631 should work better with a wider variety of sound hardware drivers. |
|
632 </p> |
|
633 |
|
634 <a href="https://cdn.netbsd.org/pub/NetBSD/misc/nia/aiomixer2.pn… |
|
635 |
|
636 <h2>Other happenings</h2> |
|
637 |
|
638 <p> |
|
639 Since I'm done plugging my own work, I thought I might talk |
|
640 a bit about some other recent changes to CURRENT. |
|
641 </p> |
|
642 |
|
643 <ul> |
|
644 <li>Most ports of NetBSD now build with GCC 10, thanks to work by … |
|
645 The new version of GCC introduced many new compiler warnings. By |
|
646 default, since NetBSD is compiled with a fixed toolchain version, |
|
647 we use <code>-Werror</code>. Many minor warnings and actual-… |
|
648 were uncovered and fixed with the new compiler.</li> |
|
649 <li>On the ARM front, support for the Allwiner V3S system-on-a-chip |
|
650 was introduced thanks to work by Rui-Xiang Guo. This is an |
|
651 older model Allwinner core, primarily used on small embedded |
|
652 devices. It's of likely interest to hardware hackers because it |
|
653 comes in an easily soldered package. A development board is |
|
654 available, the Lichee Pi Zero. Also in the Allwinner world, |
|
655 support for the H3 SoC (including the NanoPi R1) was added |
|
656 to the |
|
657 <a href="https://man.NetBSD.org/sun8icrypto.4">sun8icryp… |
|
658 driver by bad.</li> |
|
659 <li>Support for RISC-V is progressing, including an UEFI |
|
660 bootloader for 64-bit systems, and an in-kernel disassembler. |
|
661 Some NetBSD developers have recently obtained Beagle-V development |
|
662 boards.</li> |
|
663 <li>On the SPARC64 front, support for sun4v is progressing thanks |
|
664 to work by palle. The |
|
665 sun4v architecture includes most newer SPARC servers that are |
|
666 based on the |
|
667 <a href="https://en.wikipedia.org/wiki/Oracle_VM_Server_for_SPAR… |
|
668 architecture - virtualization is |
|
669 implemented at the hardware/firmware level, and operating systems |
|
670 get an abstracted view of the underlying hardware. With other |
|
671 operating systems are discussing removing support for SPARC64, there&apo… |
|
672 an interest among NetBSD developers in adding and maintaining |
|
673 support for this very interesting hardware from Oracle, Fujitsu, |
|
674 and Sun in an open source operating system, not just Oracle |
|
675 Solaris.</li> |
|
676 <li>A kernel-wide audit and rework of the auto-configuration |
|
677 APIs was completed by thorpej.</li> |
|
678 <li>Various new additions and fixes have been made to the |
|
679 networking stack's |
|
680 <a href="https://man.NetBSD.org/pppoe.4">PPP over Ethern… |
|
681 support by yamaguchi.</li> |
|
682 <li>A new API was introduced by macallan that allows adding |
|
683 a <code>-l</code> option to the |
|
684 <a href="https://man.NetBSD.org/wsfontload.8">wsfontload… |
|
685 command, allowing easy viewing of the tty fonts currently |
|
686 loaded into the kernel.</li> |
|
687 <li>... OK, I'm not done plugging my own work: I recently |
|
688 wrote new documentation on |
|
689 using |
|
690 <a href="https://www.netbsd.org/docs/guide/en/chap-virt.html&quo… |
|
691 <a href="https://www.netbsd.org/docs/guide/en/chap-power.html&qu… |
|
692 and rewrote the |
|
693 <a href="https://www.netbsd.org/docs/guide/en/index.html"&g… |
|
694 <a href="https://www.netbsd.org/docs/guide/en/chap-net-practice.… |
|
695 and |
|
696 <a href="https://www.netbsd.org/docs/guide/en/chap-audio.html&qu… |
|
697 I also recently added support for the |
|
698 <a href="https://en.wikipedia.org/wiki/Neo_(keyboard_layout)&quo… |
|
699 to NetBSD's console system - Neo 2 is a Dvorak-like |
|
700 optimized layout for German and other languages based on |
|
701 multiple layers for alphabetical characters, navigation, |
|
702 and symbols.</li> |
|
703 |
|
704 </ul> |
|
705 </content> |
|
706 </entry> |
|
707 <entry> |
|
708 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_make_system_3… |
|
709 <title type="html">GSoC Reports: Make system(3), popen(3) and po… |
|
710 <author><name>Nikita Ronja Gillmann</name></author> |
|
711 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
712 <published>2021-03-30T11:11:15+00:00</published> |
|
713 <updated>2022-02-24T10:36:27+00:00</updated> |
|
714 <category term="/The NetBSD Foundation" label="The NetBSD Founda… |
|
715 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/"… |
|
716 <summary type="html"><i>This report was prepared by Nikita… |
|
717 |
|
718 <p>This is my second and final report for the Google Summer of Cod… |
|
719 |
|
720 <p> |
|
721 My code can be found at <a href="https://github.com/teknokatze/s… |
|
722 </p> |
|
723 |
|
724 <p> |
|
725 The initial and defined goal of this project was to <a href="htt… |
|
726 For the second part I was given the task to replace fork+exec calls in o… |
|
727 </p> |
|
728 </summary> |
|
729 <content type="html"><i>This report was prepared by Nikita… |
|
730 |
|
731 <p>This is my second and final report for the Google Summer of Cod… |
|
732 |
|
733 <p> |
|
734 My code can be found at <a href="https://github.com/nikicoon/src… |
|
735 The test facilities and logs can be found in <a href="https://gi… |
|
736 A diff can be found at <a href="https://github.com/NetBSD/src/co… |
|
737 </p> |
|
738 |
|
739 <p> |
|
740 The initial and defined goal of this project was to <a href="htt… |
|
741 For the second part I was given the task to replace fork+exec calls in o… |
|
742 This second part meant in practice that I had to add and change code in … |
|
743 </p> |
|
744 |
|
745 <h2>Summary of part 1</h2> |
|
746 <p> |
|
747 Prior work: In GSoC 2012 Charles Zhang <a href="https://blog.net… |
|
748 </p> |
|
749 <p> |
|
750 After 1 week of reading POSIX and writing code, 2 weeks of coding and an… |
|
751 </p> |
|
752 <h3>system(3)</h3> |
|
753 <p>system(3) was changed to use posix_spawnattr_ (where we used si… |
|
754 <h3>popen(3) and popenve(3)</h3> |
|
755 <p> |
|
756 Since the popen and popenve implementation in NetBSD's libc use a c… |
|
757 </p> |
|
758 <p> |
|
759 pdes_child, an internal function in popen.c, now takes one more argument… |
|
760 </p> |
|
761 <p> |
|
762 On a high level what happens in pdes_child() and popen is that we first … |
|
763 </p> |
|
764 <p> |
|
765 In the new version of this helper function which now handles the majorit… |
|
766 </p> |
|
767 <p> |
|
768 The close() and dup2() actions now get replaced by corresponding file_ac… |
|
769 <p>After this series of actions, we call _readlockenv(), and call … |
|
770 <p> |
|
771 In popen and popenve our code has been reduced to the <i>pid == -1… |
|
772 </p> |
|
773 <p> |
|
774 After readlockenv we call pdes_child and pass it the command to execute … |
|
775 <p> |
|
776 The outcome of the first part is, that thanks to how we implement posix_… |
|
777 A full test with proper timing should indicate this, my reading was base… |
|
778 </p> |
|
779 <h2>sh, posix_spawn actions, libc and kernel - Part 2</h2> |
|
780 <h3>Motivation</h3> |
|
781 <p> |
|
782 The main goal of part 2 of this project was to change sh(1) to |
|
783 determine which simple cases of (v)fork + exec I could replace, and to |
|
784 replace them with posix_spawn where it makes sense.</p> |
|
785 <p> |
|
786 fork needs to create a new address space by cloning the address space, |
|
787 or in the case of vfork update at least some reference counts. |
|
788 posix_spawn can avoid most of this as it creates the new address space f… |
|
789 </p> |
|
790 <h3>Issues</h3> |
|
791 <p> |
|
792 The current posix_spawn as defined in POSIX has no good way to do tcsetp… |
|
793 that <a href="https://github.com/fish-shell/fish-shell/issues/36… |
|
794 </p> |
|
795 <h3>Implementation</h3> |
|
796 <p> |
|
797 Since, roughly speaking, modern BSDs handle &quot;#!&quot; execu… |
|
798 In pre-posix_spawn sh, every subprocess that sh (v)forked runs forkchild… |
|
799 With posix_spawn, we need to arrange posix_spawn actions to do the same … |
|
800 </p> |
|
801 <p> |
|
802 The intermediate resolution was to switch FORK_FG processes to fork+exec… |
|
803 </p> |
|
804 <pre> |
|
805 <code>int posix_spawn_file_actions_addtcsetpgrp(posix_spawn_file_a… |
|
806 </pre> |
|
807 <p> |
|
808 The kernel part of this was implemented inline in sys/kern/kern_exec.c, … |
|
809 </p> |
|
810 <h3>Future steps</h3> |
|
811 <h4>posix_spawnp kernel implementation</h4> |
|
812 <p> |
|
813 According to a conversation with kre@, the posix_spawnp() implementation… |
|
814 For some changes we might want a kernel implementation of posix_spawnp()… |
|
815 </p> |
|
816 <pre> |
|
817 <code> |
|
818 some of the file actions may be &quot;execute once only&quot;, |
|
819 they can't be repeated (eg: handling &quot;set -C; cat foo &… |
|
820 can only be created once, that has to happen before the exec (as the fd |
|
821 needs to be made stdout), and then the exec part of posix_spawn is |
|
822 attempted - if that fails, when it can't find &quot;cat&quo… |
|
823 whatever is first in $PATH) and we move along to the next entry (maybe /… |
|
824 doesn't really matter) then the repeated file action fails, as file… |
|
825 exists, and &quot;set -C&quot; demands that we cannot open an al… |
|
826 (noclobber mode). It would be nice for this if there were &quot;cl… |
|
827 failure&quot; actions, but that is likely to be very difficult to ge… |
|
828 and each would need to be attached to a file action, so only those which |
|
829 had been performed would result in cleanup attempts. |
|
830 </code> |
|
831 </pre> |
|
832 <h4>Replacing all of fork+exec in sh</h4> |
|
833 <p> |
|
834 Ideally we could replace all of (v)fork + exec with posix_spawn. |
|
835 According to my mentors there is pmap synchronisation as an impact of |
|
836 constructing the vm space from scratch with (v)fork. |
|
837 Less IPIs (inter-processor interrupts) matter for small processes too. |
|
838 </p> |
|
839 <h4>posix_spawn_file_action_ioctl</h4> |
|
840 <p> |
|
841 Future directions could involve a posix_spawn action for an arbitrary io… |
|
842 </p> |
|
843 <h2>Thanks</h2> |
|
844 <p> |
|
845 My thanks go to fellow NetBSD developers for answering questions, most r… |
|
846 </p></content> |
|
847 </entry> |
|
848 <entry> |
|
849 <id>https://blog.netbsd.org/tnf/entry/hitting_donation_milestone… |
|
850 <title type="html">Hitting donation milestone, financial report … |
|
851 <author><name>Maya Rashish</name></author> |
|
852 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
853 <published>2021-03-29T09:17:29+00:00</published> |
|
854 <updated>2021-03-29T10:32:22+00:00</updated> |
|
855 <category term="/The NetBSD Foundation" label="The NetBSD Founda… |
|
856 <summary type="html"><p> |
|
857 We nearly hit our donation milestone set after the release of 9.0 of $50… |
|
858 These donations have enabled us to fund significant paid work on NetBSD … |
|
859 </p></summary> |
|
860 <content type="html"><p> |
|
861 We nearly hit our 2020 donation milestone set after the release of 9.0 o… |
|
862 These donations have enabled us to fund significant work on NetBSD in 20… |
|
863 <ul> |
|
864 <li>an aarch64 package build server, <a href="http://victo… |
|
865 <li><a href="https://blog.netbsd.org/tnf/entry/wifi_renewa… |
|
866 <li><a href="https://blog.netbsd.org/tnf/entry/lldb_work_c… |
|
867 <li><a href="https://blog.netbsd.org/tnf/entry/accomplishm… |
|
868 </ul> |
|
869 </p> |
|
870 |
|
871 <p> |
|
872 If you are interested in seeing more work like this, please consider don… |
|
873 </p> |
|
874 |
|
875 <p> |
|
876 The <a href="https://www.netbsd.org/foundation/reports/financial… |
|
877 </p> |
|
878 |
|
879 <p> |
|
880 Note: We realize that this data is inconsistent with the website indicat… |
|
881 </p></content> |
|
882 </entry> |
|
883 <entry> |
|
884 <id>https://blog.netbsd.org/tnf/entry/allen_k_briggs_memorial_sc… |
|
885 <title type="html">Allen K. Briggs Memorial Scholarship</title> |
|
886 <author><name>Leonardo Taccari</name></author> |
|
887 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
888 <published>2020-12-21T11:37:59+00:00</published> |
|
889 <updated>2020-12-21T11:37:59+00:00</updated> |
|
890 <category term="/General" label="General" /> |
|
891 <content type="html"><p> |
|
892 Allen Briggs was one of the earliest members of the NetBSD community, |
|
893 pursuing his interest in macBSD, and moving to become a NetBSD |
|
894 developer when the two projects merged. Allen was known for his |
|
895 quiet and relaxed manner, and always brought a keen wisdom with |
|
896 him; allied with his acute technical expertise, he was one of the |
|
897 most valued members of the NetBSD community. |
|
898 </p> |
|
899 |
|
900 <p> |
|
901 He was a revered member of the NetBSD core team, and keenly involved |
|
902 in many aspects of its application; from working on ARM chips to |
|
903 helping architect many projects, Allen was renowned for his expertise. |
|
904 He was a distinguished engineer at Apple, and used his NetBSD |
|
905 expertise there to bring products to market. |
|
906 </p> |
|
907 |
|
908 <p> |
|
909 Allen lived in Blacksburg Virginia with his wife and twin boys and |
|
910 was active with various community volunteer groups. His family |
|
911 touched the families of many other NetBSD developers and those |
|
912 friendships have endured beyond his passing. |
|
913 </p> |
|
914 |
|
915 <p> |
|
916 We have received the following from Allen's family and decided to |
|
917 share it with the NetBSD community. If you can, we would ask you |
|
918 to consider contributing to his Memorial Scholarship. |
|
919 </p> |
|
920 |
|
921 <p> |
|
922 <a href="https://www.ncssm.edu/donate/distance-education/allen-k… |
|
923 </p> |
|
924 |
|
925 <p> |
|
926 The Allen K. Briggs Memorial Scholarship is an endowment to provide |
|
927 scholarships in perpetuity for summer programs at the North Carolina |
|
928 School of Science & Math, which Allen considered to be a place that |
|
929 fundamentally shaped him as a person. We would love to invite |
|
930 Allen's friends and colleagues from the BSD community to donate to |
|
931 this cause so that we can provide more scholarships to students |
|
932 with financial need each year. We are approximately halfway to our |
|
933 goal of $50K with aspirations to exceed that target and fund |
|
934 additional scholarships. |
|
935 </p> |
|
936 |
|
937 <p> |
|
938 Two quick notes on donating: <strong>Important!</strong> Whe… |
|
939 select "Allen K. Briggs Memorial Scholarship" under designation |
|
940 for the donation to be routed to the scholarship If you have the |
|
941 option to use employer matching (i.e., donating to NCSSM through |
|
942 an employer portal to secure a match from your employer), please |
|
943 email the NCSSM Foundation's Director of Development, April Horton |
|
944 (<code>[email protected]</code>), after donating to let… |
|
945 your gift and employer match to go to the Allen K. Briggs Memorial |
|
946 Scholarship Thanks in advance for your help. I'd be happy to answer |
|
947 any questions you or any others have about this. |
|
948 </p> |
|
949 </content> |
|
950 </entry> |
|
951 <entry> |
|
952 <id>https://blog.netbsd.org/tnf/entry/netbsd_9_1_released</id> |
|
953 <title type="html">NetBSD 9.1 released</title> |
|
954 <author><name>martin</name></author> |
|
955 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
956 <published>2020-10-21T04:19:23+00:00</published> |
|
957 <updated>2020-10-21T04:19:23+00:00</updated> |
|
958 <category term="/Release engineering" label="Release engineering… |
|
959 <category term="netbsd-9" scheme="http://roller.apache.org/ns/ta… |
|
960 <category term="9.1" scheme="http://roller.apache.org/ns/tags/" … |
|
961 <category term="release" scheme="http://roller.apache.org/ns/tag… |
|
962 <category term="anouncement" scheme="http://roller.apache.org/ns… |
|
963 <summary type="html"><p>NetBSD 9.1, the first maintenance … |
|
964 <content type="html"><p>After a small delay<super>&l… |
|
965 <p> |
|
966 The new release features (among various other changes) many bug fixes, |
|
967 a few performance enhancements, stability improvements for ZFS and LFS |
|
968 and support for USB security keys in a mode easily usable in Firefox |
|
969 and other applications.</p> |
|
970 <p> |
|
971 For more details and instructions see the <a href="https://www.n… |
|
972 <p> |
|
973 Get NetBSD 9.1 from our <a href="https://cdn.netbsd.org/pub/NetB… |
|
974 <p> |
|
975 Complete source and binaries for NetBSD are available for download at ma… |
|
976 |
|
977 <p style="font-size: 0.8em" name="footnote_delay"… |
|
978 </entry> |
|
979 <entry> |
|
980 <id>https://blog.netbsd.org/tnf/entry/google_summer_of_code_2020… |
|
981 <title type="html">Google Summer of Code 2020: [Final Report] En… |
|
982 <author><name>Kamil Rytarowski</name></author> |
|
983 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
984 <published>2020-10-19T13:20:28+00:00</published> |
|
985 <updated>2020-10-19T13:20:28+00:00</updated> |
|
986 <category term="/The NetBSD Foundation" label="The NetBSD Founda… |
|
987 <category term="syzkaller" scheme="http://roller.apache.org/ns/t… |
|
988 <summary type="html">This report was written by Ayushu Sharma as… |
|
989 |
|
990 <p>This post is a follow up of the <a href="https://blog.n… |
|
991 <content type="html">This report was written by Ayushu Sharma as… |
|
992 |
|
993 <p>This post is a follow up of the <a href="https://blog.n… |
|
994 |
|
995 <h2>Sys2syz</h2> |
|
996 <p>Sys2syz would give an extra edge to Syzkaller for NetBSD. It ha… |
|
997 |
|
998 <h2>A peek into Syz2syz Internals</h2> |
|
999 |
|
1000 <p>This tool parses the source code of device drivers present in C… |
|
1001 |
|
1002 <p>Python code follows 4 major steps:<p> |
|
1003 <ul> |
|
1004 <li><a href="https://github.com/ais2397/sys2syz/blob/maste… |
|
1005 <li><a href="https://github.com/ais2397/sys2syz/blob/maste… |
|
1006 <li><a href="https://github.com/ais2397/sys2syz/blob/maste… |
|
1007 <li><a href="https://github.com/ais2397/sys2syz/blob/maste… |
|
1008 </ul> |
|
1009 |
|
1010 <h3>Extraction:</h3> |
|
1011 <p>This step involves fetching the possible ioctl commands for the… |
|
1012 |
|
1013 <pre> |
|
1014 <code> |
|
1015 io = re.compile("#define\s+(.*)\s+_IO\((.*)\).*") |
|
1016 iow = re.compile("#define\s+(.*)\s+_IOW\((.*),\s+(.*),\s+(.… |
|
1017 ior = re.compile("#define\s+(.*)\s+_IOR\((.*),\s+(.*),\s+(.… |
|
1018 iowr = re.compile("#define\s+(.*)\s+_IOWR\((.*),\s+(.*),\s+… |
|
1019 </code> |
|
1020 </pre> |
|
1021 |
|
1022 <p> |
|
1023 Code scans through all the header files present in the target device fo… |
|
1024 |
|
1025 Example output: |
|
1026 <pre> |
|
1027 <code> |
|
1028 out, I2C_IOCTL_EXEC, i2c_ioctl_exec_t |
|
1029 </code> |
|
1030 </pre> |
|
1031 |
|
1032 <h3>Preprocessing:</h3> |
|
1033 <p>Preprocessing is required for getting XML files, about which w… |
|
1034 <p>Extracted commands are modified with the help of parse_commands… |
|
1035 |
|
1036 <h3>Generating XML files</h3> |
|
1037 <p>Run C2xml on the preprocessed files to fetch XML files which st… |
|
1038 <pre> |
|
1039 <code> |
|
1040 &ltsymbol type="struct" id="_5970" file=… |
|
1041 &ltsymbol type="node" id="_5971"… |
|
1042 &ltsymbol type="node" id="_5972"… |
|
1043 &ltsymbol type="node" id="_5973"… |
|
1044 &lt/symbol&gt |
|
1045 &ltsymbol type="pointer" id="_5976" file… |
|
1046 &ltsymbol type="array" id="_5978" file=&… |
|
1047 </code> |
|
1048 </pre> |
|
1049 |
|
1050 <p>We would further see how attributes like - idents, id, type, ba… |
|
1051 </p> |
|
1052 |
|
1053 <h3>Descriptions.py</h3> |
|
1054 <p>Final part, which offers a txt file storing all the required de… |
|
1055 <p>Xml files for the given target device are parsed to form trees,… |
|
1056 <pre> |
|
1057 <code> |
|
1058 for file in (os.listdir(self.target)): |
|
1059 tree = ET.parse(self.target+file) |
|
1060 self.trees.append(tree) |
|
1061 </code> |
|
1062 </pre> |
|
1063 |
|
1064 <p>We then traverse through these trees to search for the argument… |
|
1065 </p> |
|
1066 <p>Building structs and unions involves defining their elements to… |
|
1067 |
|
1068 <pre> |
|
1069 <code> |
|
1070 name = child.get("ident") |
|
1071 if name not in self.structs_and_unions.keys(): |
|
1072 elements = {} |
|
1073 for element in child: |
|
1074 elem_type = self.get_type(element) |
|
1075 elem_ident = element.get("ident") |
|
1076 if elem_type == None: |
|
1077 elem_type = element.get("type") |
|
1078 elements[element.get("ident")] = elem_type |
|
1079 |
|
1080 element_str = "" |
|
1081 for element in elements: |
|
1082 element_str += element + "\t" + elements[e… |
|
1083 self.structs_and_unions[name] = " {\n" + eleme… |
|
1084 return str(name) |
|
1085 </code> |
|
1086 </pre> |
|
1087 |
|
1088 <p>Task of creating descriptions for arrays is made simpler due to… |
|
1089 When it comes to dealing with pointers, syzkaller needs the user to fill… |
|
1090 |
|
1091 <p>There is another category named as nodes which can be distingui… |
|
1092 </p> |
|
1093 |
|
1094 <h2>Result</h2> |
|
1095 |
|
1096 <p>Once the setup script for sys2syz is executed, sys2syz can be u… |
|
1097 <pre> |
|
1098 <code>#bin/sh |
|
1099 python sys2syz.py -t &ltabsolute_path_to_device_driver_source&gt… |
|
1100 </code> |
|
1101 </pre> |
|
1102 <p>This would generate a dev_&ltdevice_driver&gt.txt file … |
|
1103 |
|
1104 <pre> |
|
1105 <code> |
|
1106 #Autogenerated by sys2syz |
|
1107 include <i2c_io.h> |
|
1108 |
|
1109 resource fd_i2c[fd] |
|
1110 |
|
1111 syz_open_dev$I2C(dev ptr[in, string["/dev/i2c"]], id intptr, f… |
|
1112 |
|
1113 ioctl$I2C_IOCTL_EXEC(fd fd_i2c, cmd const[I2C_IOCTL_EXEC], arg ptr[out, … |
|
1114 |
|
1115 i2c_ioctl_exec { |
|
1116 iie_op flags[i2c_op_t_flags] |
|
1117 iie_addr int16 |
|
1118 iie_buflen len[iie_buf, intptr] |
|
1119 iie_buf buffer[out] |
|
1120 iie_cmdlen len[iie_cmd, intptr] |
|
1121 iie_cmd buffer[out] |
|
1122 } |
|
1123 </code> |
|
1124 </pre> |
|
1125 |
|
1126 <h2>Future Work</h2> |
|
1127 <p>Though we have a basic working structure of this tool, yet a lo… |
|
1128 <p>Some other yet-to-be-done tasks include- |
|
1129 <ul> |
|
1130 <li> Generating descriptions for function type </li> |
|
1131 <li> Calculating attributes for structs and unions </li> |
|
1132 </ul> |
|
1133 |
|
1134 <h2>Summary</h2> |
|
1135 |
|
1136 <p>We have surely reached closer to our goals but the project need… |
|
1137 <p>Atlast, a word of thanks to my mentors William Coldwell, Siddha… |
|
1138 </entry> |
|
1139 <entry> |
|
1140 <id>https://blog.netbsd.org/tnf/entry/the_gnu_gdb_debugger_and4<… |
|
1141 <title type="html">The GNU GDB Debugger and NetBSD (Part 5) </ti… |
|
1142 <author><name>Kamil Rytarowski</name></author> |
|
1143 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
1144 <published>2020-10-07T17:16:53+00:00</published> |
|
1145 <updated>2020-10-07T17:24:34+00:00</updated> |
|
1146 <category term="/The NetBSD Foundation" label="The NetBSD Founda… |
|
1147 <category term="binutils" scheme="http://roller.apache.org/ns/ta… |
|
1148 <category term="gdbserver" scheme="http://roller.apache.org/ns/t… |
|
1149 <category term="gdb" scheme="http://roller.apache.org/ns/tags/" … |
|
1150 <summary type="html">The NetBSD developers maintain two copies o… |
|
1151 <ul> |
|
1152 <li>One in the base-system that includes a significant set of loca… |
|
1153 <li>Another one in pkgsrc whose patching is limited to mostly buil… |
|
1154 </ul> |
|
1155 <p> |
|
1156 The base-system version of GDB (GPLv3) still relies on local patching to… |
|
1157 I have set a goal to reduce the number of custom patches to bare minimum… |
|
1158 <content type="html">The NetBSD developers maintain two copies o… |
|
1159 <ul> |
|
1160 <li>One in the base-system that includes a significant set of loca… |
|
1161 <li>Another one in pkgsrc whose patching is limited to mostly buil… |
|
1162 </ul> |
|
1163 <p> |
|
1164 The base-system version of GDB (GPLv3) still relies on local patching to… |
|
1165 I have set a goal to reduce the number of custom patches to bare minimum… |
|
1166 <p> |
|
1167 <h2>GDB changes</h2> |
|
1168 <p> |
|
1169 Last month, the NetBSD/amd64 support was merged into gdbserver. |
|
1170 This month, the gdbserver target support was extended to |
|
1171 <a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit… |
|
1172 <a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit… |
|
1173 The gdbserver and gdb code was cleaned up, refactored and made capable o… |
|
1174 <p> |
|
1175 Meanwhile, the NetBSD/i386 build of GDB was |
|
1176 <a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit… |
|
1177 The missing include of x86-bsd-nat.h as a common header was added to i38… |
|
1178 The i386 GDB code for BSD contained a runtime assert that verified wheth… |
|
1179 <var>struct sigcontext</var> is compatible with the system h… |
|
1180 this structure since 2003, after the switch to ucontext_t, and the valid… |
|
1181 effective. After the switch to newer GCC, this was reported as a unused … |
|
1182 I have decided to <a href="https://sourceware.org/git/?p=binutil… |
|
1183 This was followed up by a small <a href="https://sourceware.org/… |
|
1184 <p> |
|
1185 The NetBSD team has noticed that the GDB's agent.cc code contains a… |
|
1186 The traditional behavior of the BSD kernel is that passing random values… |
|
1187 can cause failures. In order to prevent the problems, the sockaddr_un st… |
|
1188 I've reimplemented the fix and successfully |
|
1189 <a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit… |
|
1190 <p> |
|
1191 In order to easily resolve the issue with environment hardening enforced… |
|
1192 I've <a href="https://sourceware.org/git/?p=binutils-gdb.gi… |
|
1193 a runtime warning whenever byte transfers betweeen the debugee and debug… |
|
1194 <p> |
|
1195 <h2>binutils changes</h2> |
|
1196 <p> |
|
1197 I've added support for NetBSD/aarch64 upstream, in |
|
1198 <a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit… |
|
1199 <a href="https://sourceware.org/git/?p=binutils-gdb.git;a=commit… |
|
1200 NetBSD still carries local patches for the GNU binutils components, and … |
|
1201 <p> |
|
1202 <h2>Summary</h2> |
|
1203 <p> |
|
1204 The NetBSD support in GNU binutils and GDB is improving promptly, and th… |
|
1205 platforms of amd64, i386 and aarch64 are getting proper support out of t… |
|
1206 The remaining patches for these CPUs include: streamlining kgdb support, |
|
1207 adding native GDB support for aarch64, |
|
1208 upstreaming local modifications from the GNU binutils components (especi… |
|
1209 in the dependent projects like libiberty and gnulib. |
|
1210 Then, the remaining work is to streamline support for the remaining CPUs… |
|
1211 to develop the missing generic features (such as listing open file descr… |
|
1212 to fix failures in the regression test-suite.</content> |
|
1213 </entry> |
|
1214 <entry> |
|
1215 <id>https://blog.netbsd.org/tnf/entry/wayland_on_netbsd_trials_a… |
|
1216 <title type="html">Wayland on NetBSD - trials and tribulations</… |
|
1217 <author><name>Nia Alarie</name></author> |
|
1218 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
1219 <published>2020-09-28T17:34:22+00:00</published> |
|
1220 <updated>2020-09-28T17:34:22+00:00</updated> |
|
1221 <category term="/Ports" label="Ports" /> |
|
1222 <summary type="html"><p>After I posted about the <a hre… |
|
1223 <content type="html"><p>After I posted about the <a hre… |
|
1224 <p>Last year (and early this year) I was responsible for porting t… |
|
1225 <a href="http://ftp.netbsd.org/pub/NetBSD/misc/nia/images/waylan… |
|
1226 alt="A Wayland compositor running on my NetBSD laptop, with a few a… |
|
1227 title="A Wayland compositor running on my NetBSD laptop, with a few… |
|
1228 <h2>Difficulties</h2> |
|
1229 <p>In a Wayland system, the &quot;compositor&quot; (displa… |
|
1230 <p>Wayland does not define protocols for features X11 users expect… |
|
1231 <p>The Wayland &quot;reference implementation&quot; is a s… |
|
1232 <p>So far, all Wayland compositors but swc have a hard dependency … |
|
1233 <p>In general, Wayland is moving away from the modularity, portabi… |
|
1234 <h2>Is it ready for production?</h2> |
|
1235 <p>No, but you can play with it.</p> |
|
1236 <ul> |
|
1237 <li>swc has some remaining bugs and instability.</li> |
|
1238 <li>swc is incompatible with key applications like Firefox, but ot… |
|
1239 <li>Other popular compositors are not yet available. Alternatively… |
|
1240 <li>You need a supported GPU or SoC with kernel modesetting, since… |
|
1241 </ul> |
|
1242 <h2>Task list</h2> |
|
1243 <ul> |
|
1244 <li>Adding support for wscons to more Wayland compositors and pers… |
|
1245 <li>Persuading developers not to add hard dependencies on <code… |
|
1246 <li>Updating the NetBSD kernel DRM/KMS stack. This is a difficult … |
|
1247 <ul> |
|
1248 <li>Getting support for newer DRM versions</li> |
|
1249 <li>Getting support for atomic modesetting</li> |
|
1250 <li>Getting support for Glamor X servers (for running X11 applicat… |
|
1251 <li>Newer AMDGPU drivers, etc</li> |
|
1252 </ul> |
|
1253 </li> |
|
1254 <li>Adding support for basic (non-DRMKMS) framebuffers to a Waylan… |
|
1255 <li>Extending swc to add more features and fix bugs.</li> |
|
1256 </ul> |
|
1257 <p>I've decided to take a break from this, since it's a … |
|
1258 Right now, X11 combined with a compositor like picom or xcompmgr is the … |
|
1259 </content> |
|
1260 </entry> |
|
1261 <entry> |
|
1262 <id>https://blog.netbsd.org/tnf/entry/default_window_manager_swi… |
|
1263 <title type="html">Default window manager switched to CTWM in Ne… |
|
1264 <author><name>Nia Alarie</name></author> |
|
1265 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
1266 <published>2020-09-28T08:33:28+00:00</published> |
|
1267 <updated>2020-09-28T17:42:32+00:00</updated> |
|
1268 <category term="/General" label="General" /> |
|
1269 <summary type="html"><p> |
|
1270 For more than 20 years, NetBSD has shipped X11 with the "classic&qu… |
|
1271 </p> |
|
1272 <p> |
|
1273 In 2015, ctwm was imported, but after that no progress was made. ctwm is… |
|
1274 </p></summary> |
|
1275 <content type="html"><p> |
|
1276 For more than 20 years, NetBSD has shipped X11 with the "classic&qu… |
|
1277 </p> |
|
1278 <p> |
|
1279 In 2015, ctwm was imported, but after that no progress was made. ctwm is… |
|
1280 </p> |
|
1281 <p> |
|
1282 Recently, I've been installing NetBSD with some people in real life… |
|
1283 </p> |
|
1284 <a href="https://ftp.netbsd.org/pub/NetBSD/misc/nia/images/deskt… |
|
1285 <p> |
|
1286 We gain some nice features like an auto-generated application menu (that… |
|
1287 </p> |
|
1288 <p> |
|
1289 If you're curious about ctwm, check out the <a href="https:… |
|
1290 </p></content> |
|
1291 </entry> |
|
1292 <entry> |
|
1293 <id>https://blog.netbsd.org/tnf/entry/google_summer_of_code_2020… |
|
1294 <title type="html">Google Summer of Code 2020: [Final Report] Ru… |
|
1295 <author><name>Kamil Rytarowski</name></author> |
|
1296 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
1297 <published>2020-09-25T21:53:00+00:00</published> |
|
1298 <updated>2020-10-19T13:22:39+00:00</updated> |
|
1299 <category term="/The NetBSD Foundation" label="The NetBSD Founda… |
|
1300 <category term="rump" scheme="http://roller.apache.org/ns/tags/"… |
|
1301 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/"… |
|
1302 <category term="fuzzing" scheme="http://roller.apache.org/ns/tag… |
|
1303 <content type="html">This report was prepared by Aditya Vardhan … |
|
1304 |
|
1305 <p>This post is the third update to the project RumpKernel Syscall… |
|
1306 <p>Part1 - <a href="https://blog.netbsd.org/tnf/entry/gsoc… |
|
1307 <p>Part2 - <a href="https://blog.netbsd.org/tnf/entry/gsoc… |
|
1308 <p>The first and second coding period was entirely dedicated to fu… |
|
1309 <p>For the duration of second coding peroid we concentrated on cra… |
|
1310 <p>For the last coding period I have looked into the internals of … |
|
1311 <p><a href="https://github.com/rumpkernel/buildrump.sh&quo… |
|
1312 <p>But the rumpkernel failed to build due to some warnings and err… |
|
1313 <pre><code><span class="hljs-attribute">nbma… |
|
1314 |
|
1315 <span class="crystal">nbmake[<span class="hljs-n… |
|
1316 &gt;&gt; <span class="hljs-symbol">ERROR:</sp… |
|
1317 &gt;&gt; make /root/buildrump.sh/obj/Makefile.first dependall<… |
|
1318 </code></pre><p>Few of the similar errors were easily … |
|
1319 <p><b>To Do</b></p> |
|
1320 <ul> |
|
1321 <li>Research more on grammar definition and look into the existing… |
|
1322 <li>Integrate <a href="https://github.com/ais2397/sys2syz&… |
|
1323 </ul> |
|
1324 <p>GSoC with NetBSD has been an amazing journey throughout, in whi… |
|
1325 </entry> |
|
1326 <entry> |
|
1327 <id>https://blog.netbsd.org/tnf/entry/google_summer_of_code_2020… |
|
1328 <title type="html">Google Summer of Code 2020: [Final Report] Cu… |
|
1329 <author><name>Kamil Rytarowski</name></author> |
|
1330 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
1331 <published>2020-09-25T21:50:01+00:00</published> |
|
1332 <updated>2020-10-19T13:24:55+00:00</updated> |
|
1333 <category term="/The NetBSD Foundation" label="The NetBSD Founda… |
|
1334 <category term="curses" scheme="http://roller.apache.org/ns/tags… |
|
1335 <summary type="html">This report was prepared by Naman Jain as a… |
|
1336 <p> |
|
1337 My GSoC project under NetBSD involves the development of the test framew… |
|
1338 <p>The first report gives a brief introduction of the project and … |
|
1339 <content type="html">This report was prepared by Naman Jain as a… |
|
1340 <p> |
|
1341 My GSoC project under NetBSD involves the development of the test framew… |
|
1342 <p>The first report gives a brief introduction of the project and … |
|
1343 <p>This being the final report in the series, I would love to shar… |
|
1344 <h2><a id="user-content-challenges-and-caveats" class… |
|
1345 <p>By the time my application for GSoC was submitted, I had gained… |
|
1346 <h2><a id="user-content-learnings" class="anchor… |
|
1347 <p>The foremost learning is from the experience of interacting wit… |
|
1348 <p>The project-specific learning was not limited to test-framework… |
|
1349 <h2><a id="user-content-some-tests-from-test-suite" c… |
|
1350 <p>In this section, I would discuss a couple of tests of the test … |
|
1351 <h3><a id="user-content-keypad-processing" class=&quo… |
|
1352 <p>An application can enable or disable the tarnslation of keypad … |
|
1353 <pre><code>include window |
|
1354 call $FALSE is_keypad $win1 |
|
1355 input "\eOA" |
|
1356 call 0x1b wgetch $win1 |
|
1357 call OK keypad $win1 $TRUE |
|
1358 input "\eOA" |
|
1359 call $KEY_UP wgetch $win1 |
|
1360 |
|
1361 # disable assembly of KEY_UP |
|
1362 call OK keyok $KEY_UP $FALSE |
|
1363 input "\eOA" |
|
1364 call 0x1b wgetch $win1 |
|
1365 </code></pre> |
|
1366 <p>As keypad translation is disabled by default, on input of &apos… |
|
1367 <h3><a id="user-content-input-mode" class="ancho… |
|
1368 <p>Curses lets the application control the effect of input using f… |
|
1369 <pre><code>include start |
|
1370 delay 1000 |
|
1371 # input delay 1000 equals to 10 tenths of seconds |
|
1372 # getch must fail for halfdelay(5) and pass for halfdelay(15) |
|
1373 input "a" |
|
1374 call OK halfdelay 15 |
|
1375 call 0x61 getch |
|
1376 call OK halfdelay 5 |
|
1377 input "a" |
|
1378 call -1 getch |
|
1379 </code></pre> |
|
1380 <p>We have set the delay for feeding input to terminal with delay … |
|
1381 <h2><a id="user-content-project-work" class="anc… |
|
1382 <p>The <a href="https://github.com/NamanJain8/curses"… |
|
1383 <p>This project involved:</p> |
|
1384 <ol> |
|
1385 <li>Improvement in testframework: |
|
1386 <ul> |
|
1387 <li>Automation of the checkfile generation.</li> |
|
1388 <li>Enhnacement of support for complex character</li> |
|
1389 <li>Addition of small features and code refactoring</li> |
|
1390 </ul> |
|
1391 </li> |
|
1392 <li>Testing and bug reports: |
|
1393 <ul> |
|
1394 <li>Tests for a family of routines like wide character, complex ch… |
|
1395 <li>Raising a bunch of Problem Report (PR) under <code>lib&l… |
|
1396 </ul> |
|
1397 </li> |
|
1398 </ol> |
|
1399 <h2><a id="user-content-future-work" class="anch… |
|
1400 <ul> |
|
1401 <li>The current testframe supports complex character, but the supp… |
|
1402 <li>Some of the tests for teminal manipulation routines like <c… |
|
1403 <li>Not specifically related to the framework, but the documentati… |
|
1404 </ul> |
|
1405 <h2><a id="user-content-acknowledgements" class="… |
|
1406 <p>I want to extend my heartfelt gratitude to my mentor Mr. Brett … |
|
1407 </entry> |
|
1408 <entry> |
|
1409 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_benchmarking_… |
|
1410 <title type="html">GSoC Reports: Benchmarking NetBSD, third eval… |
|
1411 <author><name>Leonardo Taccari</name></author> |
|
1412 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
1413 <published>2020-09-12T08:29:06+00:00</published> |
|
1414 <updated>2020-09-12T08:29:06+00:00</updated> |
|
1415 <category term="/General" label="General" /> |
|
1416 <category term="gsoc2020" scheme="http://roller.apache.org/ns/ta… |
|
1417 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/"… |
|
1418 <category term="benchmarks" scheme="http://roller.apache.org/ns/… |
|
1419 <category term="pkgsrc" scheme="http://roller.apache.org/ns/tags… |
|
1420 <summary type="html"><p>This report was written by Apurva … |
|
1421 |
|
1422 <p>This blog post is in continuation of |
|
1423 <a href="//blog.NetBSD.org/tnf/entry/gsoc_reports_benchmarking_n… |
|
1424 and <a href="//blog.NetBSD.org/tnf/entry/gsoc_reports_benchmarki… |
|
1425 blogs, and describes my progress in the final phase of GSoC 2020 under |
|
1426 The NetBSD Foundation.</p> |
|
1427 <p>In the third phase, I upgraded to the latest stable version |
|
1428 <a href="https://www.phoronix-test-suite.com/">Phoronix … |
|
1429 pkgsrc-wip, resolved the TODOs and created patches for more |
|
1430 test-profiles to fix their installation and runtime errors on |
|
1431 NetBSD-current.</p></summary> |
|
1432 <content type="html"><p>This report was written by Apurva … |
|
1433 |
|
1434 <h2>Introduction</h2> |
|
1435 <p>This blog post is in continuation of |
|
1436 <a href="//blog.NetBSD.org/tnf/entry/gsoc_reports_benchmarking_n… |
|
1437 and <a href="//blog.NetBSD.org/tnf/entry/gsoc_reports_benchmarki… |
|
1438 blogs, and describes my progress in the final phase of GSoC 2020 under |
|
1439 The NetBSD Foundation.</p> |
|
1440 <p>In the third phase, I upgraded to the latest stable version |
|
1441 <a href="https://www.phoronix-test-suite.com/">Phoronix … |
|
1442 pkgsrc-wip, resolved the TODOs and created patches for more |
|
1443 test-profiles to fix their installation and runtime errors on |
|
1444 NetBSD-current.</p> |
|
1445 |
|
1446 <h2>Progress in the third phase of GSoC</h2> |
|
1447 <h3>wip/phoronix-test-suite TODO and update</h3> |
|
1448 <p>As a newer stable version of the Phoronix Test Suite was availa… |
|
1449 upstream, I upgraded the Phoronix Test Suite from version 9.6.1 to |
|
1450 9.8.0 in pkgsrc-wip and is available as |
|
1451 <a href="https://pkgsrc.se/wip/phoronix-test-suite">wip/… |
|
1452 You can have a look at the |
|
1453 <a href="https://github.com/phoronix-test-suite/phoronix-test-su… |
|
1454 to know about the improvements between these two versions.</p> |
|
1455 <p>To get the package ready for merge in pkgsrc upstream, I also r… |
|
1456 the pkgsrc-wip TODOs.</p> |
|
1457 <h4>pkgsrc-wip commits:</h4> |
|
1458 <ul> |
|
1459 <li><a href="https://github.com/NetBSD/pkgsrc-wip/commit/7… |
|
1460 <li><a href="https://github.com/NetBSD/pkgsrc-wip/commit/7… |
|
1461 </ul> |
|
1462 <p>If any new problems are encountered, please document them in |
|
1463 <code>wip/phoronix-test-suite/TODO</code> file and/or contac… |
|
1464 <h3>Testing of automated benchmarking framework</h3> |
|
1465 <p>I had been assigned a remote testing machine having Intel 6138 … |
|
1466 processor, 40 cores, 80 threads, 192GB of RAM. I spent time reproducing |
|
1467 my automated framework i.e., Phoromatic-Anita Integration on the |
|
1468 machine. I was able to reproduce the integration framework working |
|
1469 without networking configuration, but the network bridge needs to be |
|
1470 setup on the remote machine and the integration script to be tested |
|
1471 with it. I shall continue this task in the post-GSoC period.</p> |
|
1472 <h4>Benchmarking Results</h4> |
|
1473 <p>I also performed benchmarking of NetBSD-9 amd64 native installa… |
|
1474 running 50 test-profiles on a remote machine assigned to me by mentors |
|
1475 and uploaded the benchmark results to |
|
1476 <a href="https://openbenchmarking.org/">OpenBenchmarking… |
|
1477 <ul> |
|
1478 <li><a href="https://openbenchmarking.org/result/2008287-N… |
|
1479 </ul> |
|
1480 <h3>Test-profile debugging</h3> |
|
1481 <p>I then continued the task of maintaining/porting test-profiles … |
|
1482 fixed the following test-profiles:</p> |
|
1483 <h4>Timed FFmpeg Compilation</h4> |
|
1484 <p>This test times how long it takes to build FFmpeg. |
|
1485 This test is part of <code>Processor Test</code> category.&l… |
|
1486 <h5>Original Test-profile:</h5> |
|
1487 <p><a href="https://openbenchmarking.org/test/pts/build-ff… |
|
1488 <h5>Patched Test-profile:</h5> |
|
1489 <p><a href="https://github.com/apurvanandan1997/pts-test-p… |
|
1490 <h6>Commit:</h6> |
|
1491 <ul> |
|
1492 <li><a href="https://github.com/apurvanandan1997/pts-test-… |
|
1493 </ul> |
|
1494 <h4>Compile Bench</h4> |
|
1495 <p>Compilebench tries to age a filesystem by simulating some of th… |
|
1496 IO common in creating, compiling, patching, stating and reading kernel |
|
1497 trees. It indirectly measures how well filesystems can maintain |
|
1498 directory locality as the disk fills up and directories age. |
|
1499 This test is part of <code>Disk Test</code> category.</p&… |
|
1500 <h5>Original Test-profile:</h5> |
|
1501 <p><a href="https://openbenchmarking.org/test/pts/compileb… |
|
1502 <h5>Patched Test-profile:</h5> |
|
1503 <p><a href="https://github.com/apurvanandan1997/pts-test-p… |
|
1504 <h6>Commit:</h6> |
|
1505 <ul> |
|
1506 <li><a href="https://github.com/apurvanandan1997/pts-test-… |
|
1507 </ul> |
|
1508 <h4>Timed MAFFT Alignment</h4> |
|
1509 <p>This test performs an alignment of 100 pyruvate decarboxylase s… |
|
1510 This test is part of <code>Processor Test</code> category.&l… |
|
1511 <h5>Original Test-profile:</h5> |
|
1512 <p><a href="https://openbenchmarking.org/test/pts/mafft&qu… |
|
1513 <h5>Patched Test-profile:</h5> |
|
1514 <p><a href="https://github.com/apurvanandan1997/pts-test-p… |
|
1515 <h6>Commits:</h6> |
|
1516 <ul> |
|
1517 <li><a href="https://github.com/apurvanandan1997/pts-test-… |
|
1518 <li><a href="https://github.com/apurvanandan1997/pts-test-… |
|
1519 </ul> |
|
1520 <h2>Future Plans</h2> |
|
1521 <p>This officially summarizes my GSoC project: Benchmark NetBSD, a… |
|
1522 end goal of the project that is to integrate Phoronix Test Suite with |
|
1523 NetBSD and Anita for automated benchmarking is complete and its |
|
1524 deployment on benchmark.NetBSD.org will be continued to be worked on |
|
1525 with the coordination of moderators and merging the wip of Phoronix |
|
1526 Test Suite 9.8.0 will be done by the pkgsrc maintainers in next days.<… |
|
1527 <p>I want to thank my mentors and the NetBSD community without who… |
|
1528 constant support I wouldn't have achieved the goals.</p></con… |
|
1529 </entry> |
|
1530 <entry> |
|
1531 <id>https://blog.netbsd.org/tnf/entry/the_gnu_gdb_debugger_and3<… |
|
1532 <title type="html">The GNU GDB Debugger and NetBSD (Part 4)</tit… |
|
1533 <author><name>Kamil Rytarowski</name></author> |
|
1534 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
1535 <published>2020-09-10T21:13:01+00:00</published> |
|
1536 <updated>2020-09-10T21:17:53+00:00</updated> |
|
1537 <category term="/The NetBSD Foundation" label="The NetBSD Founda… |
|
1538 <category term="gdbserver" scheme="http://roller.apache.org/ns/t… |
|
1539 <category term="gdb" scheme="http://roller.apache.org/ns/tags/" … |
|
1540 <summary type="html">The NetBSD team of developers maintains two… |
|
1541 <ul> |
|
1542 <li>One in the base-system with a stack of local patches.</li&g… |
|
1543 <li>One in pkgsrc with mostly build fix patches.</li> |
|
1544 </ul> |
|
1545 <p> |
|
1546 The base-system version of GDB (GPLv3) still relies on a set of local pa… |
|
1547 I set a goal to reduce the local patches to bare minimum, ideally reachi… |
|
1548 <content type="html">The NetBSD team of developers maintains two… |
|
1549 <ul> |
|
1550 <li>One in the base-system with a stack of local patches.</li&g… |
|
1551 <li>One in pkgsrc with mostly build fix patches.</li> |
|
1552 </ul> |
|
1553 <p> |
|
1554 The base-system version of GDB (GPLv3) still relies on a set of local pa… |
|
1555 I set a goal to reduce the local patches to bare minimum, ideally reachi… |
|
1556 <p> |
|
1557 <h1>GDB changes</h1> |
|
1558 <p> |
|
1559 Over the past month I worked on gdbserver for NetBSD/amd64 and finally |
|
1560 <a href="https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.gi… |
|
1561 <p> |
|
1562 What is gdbserver? Let's quote the official |
|
1563 <a href="https://sourceware.org/gdb/onlinedocs/gdb/Server.html&q… |
|
1564 <p> |
|
1565 <i><quote> |
|
1566 gdbserver is a control program for Unix-like systems, which allows you t… |
|
1567 <p> |
|
1568 gdbserver is not a complete replacement for the debugging stubs, because… |
|
1569 <p> |
|
1570 GDB and gdbserver communicate via either a serial line or a TCP connecti… |
|
1571 </quote></i> |
|
1572 <p> |
|
1573 This illustrated that gdbserver is especially useful for debugging appli… |
|
1574 a controlling computer equipped with full distribution sources, toolchai… |
|
1575 Eventually, this approach of gdb and gdbserver can replace the native gd… |
|
1576 debugging sessions using this protocol. |
|
1577 This design decision was already introduced into LLDB, where remote proc… |
|
1578 on Linux, NetBSD and highly recommended for other kernels. |
|
1579 <p> |
|
1580 I've picked amd64 as the first target as it's the easiest to d… |
|
1581 <p> |
|
1582 An example debugging session looks like this: |
|
1583 <pre> |
|
1584 $ uname -rms |
|
1585 NetBSD 9.99.72 amd64 |
|
1586 $ LC_ALL=C date |
|
1587 Thu Sep 10 22:43:10 CEST 2020 |
|
1588 $ ./gdbserver/gdbserver --version |
|
1589 GNU gdbserver (GDB) 10.0.50.20200910-git |
|
1590 Copyright (C) 2020 Free Software Foundation, Inc. |
|
1591 gdbserver is free software, covered by the GNU General Public License. |
|
1592 This gdbserver was configured as "x86_64-unknown-netbsd9.99" |
|
1593 $ ./gdbserver/gdbserver localhost:1234 /usr/bin/nslookup |
|
1594 Process /usr/bin/nslookup created; pid = 26383 |
|
1595 Listening on port 1234 |
|
1596 </pre> |
|
1597 <p> |
|
1598 Then on the other terminal: |
|
1599 <p> |
|
1600 <pre> |
|
1601 $ ./gdb/gdb |
|
1602 GNU gdb (GDB) 10.0.50.20200910-git |
|
1603 Copyright (C) 2020 Free Software Foundation, Inc. |
|
1604 License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/g… |
|
1605 This is free software: you are free to change and redistribute it. |
|
1606 There is NO WARRANTY, to the extent permitted by law. |
|
1607 Type "show copying" and "show warranty" for details. |
|
1608 This GDB was configured as "x86_64-unknown-netbsd9.99". |
|
1609 Type "show configuration" for configuration details. |
|
1610 For bug reporting instructions, please see: |
|
1611 <https://www.gnu.org/software/gdb/bugs/>. |
|
1612 Find the GDB manual and other documentation resources online at: |
|
1613 --Type <RET> for more, q to quit, c to continue without paging-- |
|
1614 <http://www.gnu.org/software/gdb/documentation/>. |
|
1615 |
|
1616 For help, type "help". |
|
1617 Type "apropos word" to search for commands related to "wo… |
|
1618 (gdb) target remote localhost:1234 |
|
1619 Remote debugging using localhost:1234 |
|
1620 Reading /usr/bin/nslookup from remote target... |
|
1621 warning: File transfers from remote targets can be slow. Use "set s… |
|
1622 Reading /usr/bin/nslookup from remote target... |
|
1623 Reading symbols from target:/usr/bin/nslookup... |
|
1624 Reading /usr/bin/nslookup.debug from remote target... |
|
1625 Reading /usr/bin/.debug/nslookup.debug from remote target... |
|
1626 Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target... |
|
1627 Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target... |
|
1628 Reading symbols from target:/usr/libdata/debug//usr/bin/nslookup.debug... |
|
1629 process 28353 is executing new program: /usr/bin/nslookup |
|
1630 Reading /usr/bin/nslookup from remote target... |
|
1631 Reading /usr/bin/nslookup from remote target... |
|
1632 Reading /usr/bin/nslookup.debug from remote target... |
|
1633 Reading /usr/bin/.debug/nslookup.debug from remote target... |
|
1634 Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target... |
|
1635 Reading /usr/libdata/debug//usr/bin/nslookup.debug from remote target... |
|
1636 Reading /usr/libexec/ld.elf_so from remote target... |
|
1637 Reading /usr/libexec/ld.elf_so from remote target... |
|
1638 Reading /usr/libexec/ld.elf_so.debug from remote target... |
|
1639 Reading /usr/libexec/.debug/ld.elf_so.debug from remote target... |
|
1640 Reading /usr/libdata/debug//usr/libexec/ld.elf_so.debug from remote targ… |
|
1641 Reading /usr/libdata/debug//usr/libexec/ld.elf_so.debug from remote targ… |
|
1642 warning: Invalid remote reply: timeout [kamil: repeated multiple times..… |
|
1643 Reading /usr/lib/libbind9.so.15 from remote target... |
|
1644 Reading /usr/lib/libisccfg.so.15 from remote target... |
|
1645 Reading /usr/lib/libdns.so.15 from remote target... |
|
1646 Reading /usr/lib/libns.so.15 from remote target... |
|
1647 Reading /usr/lib/libirs.so.15 from remote target... |
|
1648 Reading /usr/lib/libisccc.so.15 from remote target... |
|
1649 Reading /usr/lib/libisc.so.15 from remote target... |
|
1650 Reading /usr/lib/libkvm.so.6 from remote target... |
|
1651 Reading /usr/lib/libz.so.1 from remote target... |
|
1652 Reading /usr/lib/libblocklist.so.0 from remote target... |
|
1653 Reading /usr/lib/libpthread.so.1 from remote target... |
|
1654 Reading /usr/lib/libpthread.so.1.4.debug from remote target... |
|
1655 Reading /usr/lib/.debug/libpthread.so.1.4.debug from remote target... |
|
1656 Reading /usr/libdata/debug//usr/lib/libpthread.so.1.4.debug from remote … |
|
1657 Reading /usr/libdata/debug//usr/lib/libpthread.so.1.4.debug from remote … |
|
1658 Reading /usr/lib/libgssapi.so.11 from remote target... |
|
1659 Reading /usr/lib/libheimntlm.so.5 from remote target... |
|
1660 Reading /usr/lib/libkrb5.so.27 from remote target... |
|
1661 Reading /usr/lib/libcom_err.so.8 from remote target... |
|
1662 Reading /usr/lib/libhx509.so.6 from remote target... |
|
1663 Reading /usr/lib/libcrypto.so.14 from remote target... |
|
1664 Reading /usr/lib/libasn1.so.10 from remote target... |
|
1665 Reading /usr/lib/libwind.so.1 from remote target... |
|
1666 Reading /usr/lib/libheimbase.so.2 from remote target... |
|
1667 Reading /usr/lib/libroken.so.20 from remote target... |
|
1668 Reading /usr/lib/libsqlite3.so.1 from remote target... |
|
1669 Reading /usr/lib/libcrypt.so.1 from remote target... |
|
1670 Reading /usr/lib/libutil.so.7 from remote target... |
|
1671 Reading /usr/lib/libedit.so.3 from remote target... |
|
1672 Reading /usr/lib/libterminfo.so.2 from remote target... |
|
1673 Reading /usr/lib/libc.so.12 from remote target... |
|
1674 Reading /usr/lib/libgcc_s.so.1 from remote target... |
|
1675 Reading symbols from target:/usr/lib/libbind9.so.15... |
|
1676 Reading /usr/lib/libbind9.so.15.0.debug from remote target... |
|
1677 Reading /usr/lib/.debug/libbind9.so.15.0.debug from remote target... |
|
1678 Reading /usr/libdata/debug//usr/lib/libbind9.so.15.0.debug from remote t… |
|
1679 Reading /usr/libdata/debug//usr/lib/libbind9.so.15.0.debug from remote t… |
|
1680 Reading symbols from target:/usr/libdata/debug//usr/lib/libbind9.so.15.0… |
|
1681 Reading symbols from target:/usr/lib/libisccfg.so.15... |
|
1682 Reading /usr/lib/libisccfg.so.15.0.debug from remote target... |
|
1683 Reading /usr/lib/.debug/libisccfg.so.15.0.debug from remote target... |
|
1684 Reading /usr/libdata/debug//usr/lib/libisccfg.so.15.0.debug from remote … |
|
1685 Reading /usr/libdata/debug//usr/lib/libisccfg.so.15.0.debug from remote … |
|
1686 --Type <RET> for more, q to quit, c to continue without paging-- |
|
1687 Reading symbols from target:/usr/libdata/debug//usr/lib/libisccfg.so.15.… |
|
1688 Reading symbols from target:/usr/lib/libdns.so.15... |
|
1689 Reading /usr/lib/libdns.so.15.0.debug from remote target... |
|
1690 Reading /usr/lib/.debug/libdns.so.15.0.debug from remote target... |
|
1691 Reading /usr/libdata/debug//usr/lib/libdns.so.15.0.debug from remote tar… |
|
1692 Reading /usr/libdata/debug//usr/lib/libdns.so.15.0.debug from remote tar… |
|
1693 Reading symbols from target:/usr/libdata/debug//usr/lib/libdns.so.15.0.d… |
|
1694 Reading symbols from target:/usr/lib/libns.so.15... |
|
1695 Reading /usr/lib/libns.so.15.0.debug from remote target... |
|
1696 Reading /usr/lib/.debug/libns.so.15.0.debug from remote target... |
|
1697 Reading /usr/libdata/debug//usr/lib/libns.so.15.0.debug from remote targ… |
|
1698 Reading /usr/libdata/debug//usr/lib/libns.so.15.0.debug from remote targ… |
|
1699 Reading symbols from target:/usr/libdata/debug//usr/lib/libns.so.15.0.de… |
|
1700 Reading symbols from target:/usr/lib/libirs.so.15... |
|
1701 Reading /usr/lib/libirs.so.15.0.debug from remote target... |
|
1702 Reading /usr/lib/.debug/libirs.so.15.0.debug from remote target... |
|
1703 Reading /usr/libdata/debug//usr/lib/libirs.so.15.0.debug from remote tar… |
|
1704 Reading /usr/libdata/debug//usr/lib/libirs.so.15.0.debug from remote tar… |
|
1705 Reading symbols from target:/usr/libdata/debug//usr/lib/libirs.so.15.0.d… |
|
1706 Reading symbols from target:/usr/lib/libisccc.so.15... |
|
1707 Reading /usr/lib/libisccc.so.15.0.debug from remote target... |
|
1708 Reading /usr/lib/.debug/libisccc.so.15.0.debug from remote target... |
|
1709 Reading /usr/libdata/debug//usr/lib/libisccc.so.15.0.debug from remote t… |
|
1710 Reading /usr/libdata/debug//usr/lib/libisccc.so.15.0.debug from remote t… |
|
1711 Reading symbols from target:/usr/libdata/debug//usr/lib/libisccc.so.15.0… |
|
1712 Reading symbols from target:/usr/lib/libisc.so.15... |
|
1713 Reading /usr/lib/libisc.so.15.0.debug from remote target... |
|
1714 Reading /usr/lib/.debug/libisc.so.15.0.debug from remote target... |
|
1715 Reading /usr/libdata/debug//usr/lib/libisc.so.15.0.debug from remote tar… |
|
1716 Reading /usr/libdata/debug//usr/lib/libisc.so.15.0.debug from remote tar… |
|
1717 Reading symbols from target:/usr/libdata/debug//usr/lib/libisc.so.15.0.d… |
|
1718 Reading symbols from target:/usr/lib/libkvm.so.6... |
|
1719 Reading /usr/lib/libkvm.so.6.0.debug from remote target... |
|
1720 Reading /usr/lib/.debug/libkvm.so.6.0.debug from remote target... |
|
1721 Reading /usr/libdata/debug//usr/lib/libkvm.so.6.0.debug from remote targ… |
|
1722 Reading /usr/libdata/debug//usr/lib/libkvm.so.6.0.debug from remote targ… |
|
1723 Reading symbols from target:/usr/libdata/debug//usr/lib/libkvm.so.6.0.de… |
|
1724 Reading symbols from target:/usr/lib/libz.so.1... |
|
1725 Reading /usr/lib/libz.so.1.0.debug from remote target... |
|
1726 Reading /usr/lib/.debug/libz.so.1.0.debug from remote target... |
|
1727 Reading /usr/libdata/debug//usr/lib/libz.so.1.0.debug from remote target… |
|
1728 Reading /usr/libdata/debug//usr/lib/libz.so.1.0.debug from remote target… |
|
1729 Reading symbols from target:/usr/libdata/debug//usr/lib/libz.so.1.0.debu… |
|
1730 Reading symbols from target:/usr/lib/libblocklist.so.0... |
|
1731 Reading /usr/lib/libblocklist.so.0.0.debug from remote target... |
|
1732 Reading /usr/lib/.debug/libblocklist.so.0.0.debug from remote target... |
|
1733 Reading /usr/libdata/debug//usr/lib/libblocklist.so.0.0.debug from remot… |
|
1734 Reading /usr/libdata/debug//usr/lib/libblocklist.so.0.0.debug from remot… |
|
1735 Reading symbols from target:/usr/libdata/debug//usr/lib/libblocklist.so.… |
|
1736 Reading symbols from target:/usr/lib/libgssapi.so.11... |
|
1737 Reading /usr/lib/libgssapi.so.11.0.debug from remote target... |
|
1738 Reading /usr/lib/.debug/libgssapi.so.11.0.debug from remote target... |
|
1739 Reading /usr/libdata/debug//usr/lib/libgssapi.so.11.0.debug from remote … |
|
1740 Reading /usr/libdata/debug//usr/lib/libgssapi.so.11.0.debug from remote … |
|
1741 Reading symbols from target:/usr/libdata/debug//usr/lib/libgssapi.so.11.… |
|
1742 Reading symbols from target:/usr/lib/libheimntlm.so.5... |
|
1743 Reading /usr/lib/libheimntlm.so.5.0.debug from remote target... |
|
1744 Reading /usr/lib/.debug/libheimntlm.so.5.0.debug from remote target... |
|
1745 Reading /usr/libdata/debug//usr/lib/libheimntlm.so.5.0.debug from remote… |
|
1746 Reading /usr/libdata/debug//usr/lib/libheimntlm.so.5.0.debug from remote… |
|
1747 Reading symbols from target:/usr/libdata/debug//usr/lib/libheimntlm.so.5… |
|
1748 Reading symbols from target:/usr/lib/libkrb5.so.27... |
|
1749 Reading /usr/lib/libkrb5.so.27.0.debug from remote target... |
|
1750 Reading /usr/lib/.debug/libkrb5.so.27.0.debug from remote target... |
|
1751 Reading /usr/libdata/debug//usr/lib/libkrb5.so.27.0.debug from remote ta… |
|
1752 Reading /usr/libdata/debug//usr/lib/libkrb5.so.27.0.debug from remote ta… |
|
1753 Reading symbols from target:/usr/libdata/debug//usr/lib/libkrb5.so.27.0.… |
|
1754 Reading symbols from target:/usr/lib/libcom_err.so.8... |
|
1755 Reading /usr/lib/libcom_err.so.8.0.debug from remote target... |
|
1756 Reading /usr/lib/.debug/libcom_err.so.8.0.debug from remote target... |
|
1757 Reading /usr/libdata/debug//usr/lib/libcom_err.so.8.0.debug from remote … |
|
1758 Reading /usr/libdata/debug//usr/lib/libcom_err.so.8.0.debug from remote … |
|
1759 Reading symbols from target:/usr/libdata/debug//usr/lib/libcom_err.so.8.… |
|
1760 Reading symbols from target:/usr/lib/libhx509.so.6... |
|
1761 Reading /usr/lib/libhx509.so.6.0.debug from remote target... |
|
1762 Reading /usr/lib/.debug/libhx509.so.6.0.debug from remote target... |
|
1763 Reading /usr/libdata/debug//usr/lib/libhx509.so.6.0.debug from remote ta… |
|
1764 Reading /usr/libdata/debug//usr/lib/libhx509.so.6.0.debug from remote ta… |
|
1765 Reading symbols from target:/usr/libdata/debug//usr/lib/libhx509.so.6.0.… |
|
1766 Reading symbols from target:/usr/lib/libcrypto.so.14... |
|
1767 Reading /usr/lib/libcrypto.so.14.0.debug from remote target... |
|
1768 Reading /usr/lib/.debug/libcrypto.so.14.0.debug from remote target... |
|
1769 Reading /usr/libdata/debug//usr/lib/libcrypto.so.14.0.debug from remote … |
|
1770 Reading /usr/libdata/debug//usr/lib/libcrypto.so.14.0.debug from remote … |
|
1771 Reading symbols from target:/usr/libdata/debug//usr/lib/libcrypto.so.14.… |
|
1772 Reading symbols from target:/usr/lib/libasn1.so.10... |
|
1773 Reading /usr/lib/libasn1.so.10.0.debug from remote target... |
|
1774 Reading /usr/lib/.debug/libasn1.so.10.0.debug from remote target... |
|
1775 Reading /usr/libdata/debug//usr/lib/libasn1.so.10.0.debug from remote ta… |
|
1776 Reading /usr/libdata/debug//usr/lib/libasn1.so.10.0.debug from remote ta… |
|
1777 Reading symbols from target:/usr/libdata/debug//usr/lib/libasn1.so.10.0.… |
|
1778 Reading symbols from target:/usr/lib/libwind.so.1... |
|
1779 Reading /usr/lib/libwind.so.1.0.debug from remote target... |
|
1780 Reading /usr/lib/.debug/libwind.so.1.0.debug from remote target... |
|
1781 Reading /usr/libdata/debug//usr/lib/libwind.so.1.0.debug from remote tar… |
|
1782 Reading /usr/libdata/debug//usr/lib/libwind.so.1.0.debug from remote tar… |
|
1783 Reading symbols from target:/usr/libdata/debug//usr/lib/libwind.so.1.0.d… |
|
1784 Reading symbols from target:/usr/lib/libheimbase.so.2... |
|
1785 Reading /usr/lib/libheimbase.so.2.0.debug from remote target... |
|
1786 Reading /usr/lib/.debug/libheimbase.so.2.0.debug from remote target... |
|
1787 Reading /usr/libdata/debug//usr/lib/libheimbase.so.2.0.debug from remote… |
|
1788 Reading /usr/libdata/debug//usr/lib/libheimbase.so.2.0.debug from remote… |
|
1789 Reading symbols from target:/usr/libdata/debug//usr/lib/libheimbase.so.2… |
|
1790 Reading symbols from target:/usr/lib/libroken.so.20... |
|
1791 Reading /usr/lib/libroken.so.20.0.debug from remote target... |
|
1792 Reading /usr/lib/.debug/libroken.so.20.0.debug from remote target... |
|
1793 Reading /usr/libdata/debug//usr/lib/libroken.so.20.0.debug from remote t… |
|
1794 Reading /usr/libdata/debug//usr/lib/libroken.so.20.0.debug from remote t… |
|
1795 Reading symbols from target:/usr/libdata/debug//usr/lib/libroken.so.20.0… |
|
1796 Reading symbols from target:/usr/lib/libsqlite3.so.1... |
|
1797 Reading /usr/lib/libsqlite3.so.1.4.debug from remote target... |
|
1798 Reading /usr/lib/.debug/libsqlite3.so.1.4.debug from remote target... |
|
1799 Reading /usr/libdata/debug//usr/lib/libsqlite3.so.1.4.debug from remote … |
|
1800 Reading /usr/libdata/debug//usr/lib/libsqlite3.so.1.4.debug from remote … |
|
1801 Reading symbols from target:/usr/libdata/debug//usr/lib/libsqlite3.so.1.… |
|
1802 Reading symbols from target:/usr/lib/libcrypt.so.1... |
|
1803 Reading /usr/lib/libcrypt.so.1.0.debug from remote target... |
|
1804 Reading /usr/lib/.debug/libcrypt.so.1.0.debug from remote target... |
|
1805 Reading /usr/libdata/debug//usr/lib/libcrypt.so.1.0.debug from remote ta… |
|
1806 Reading /usr/libdata/debug//usr/lib/libcrypt.so.1.0.debug from remote ta… |
|
1807 Reading symbols from target:/usr/libdata/debug//usr/lib/libcrypt.so.1.0.… |
|
1808 Reading symbols from target:/usr/lib/libutil.so.7... |
|
1809 Reading /usr/lib/libutil.so.7.24.debug from remote target... |
|
1810 Reading /usr/lib/.debug/libutil.so.7.24.debug from remote target... |
|
1811 Reading /usr/libdata/debug//usr/lib/libutil.so.7.24.debug from remote ta… |
|
1812 Reading /usr/libdata/debug//usr/lib/libutil.so.7.24.debug from remote ta… |
|
1813 Reading symbols from target:/usr/libdata/debug//usr/lib/libutil.so.7.24.… |
|
1814 Reading symbols from target:/usr/lib/libedit.so.3... |
|
1815 Reading /usr/lib/libedit.so.3.1.debug from remote target... |
|
1816 Reading /usr/lib/.debug/libedit.so.3.1.debug from remote target... |
|
1817 Reading /usr/libdata/debug//usr/lib/libedit.so.3.1.debug from remote tar… |
|
1818 Reading /usr/libdata/debug//usr/lib/libedit.so.3.1.debug from remote tar… |
|
1819 Reading symbols from target:/usr/libdata/debug//usr/lib/libedit.so.3.1.d… |
|
1820 Reading symbols from target:/usr/lib/libterminfo.so.2... |
|
1821 Reading /usr/lib/libterminfo.so.2.0.debug from remote target... |
|
1822 Reading /usr/lib/.debug/libterminfo.so.2.0.debug from remote target... |
|
1823 Reading /usr/libdata/debug//usr/lib/libterminfo.so.2.0.debug from remote… |
|
1824 Reading /usr/libdata/debug//usr/lib/libterminfo.so.2.0.debug from remote… |
|
1825 Reading symbols from target:/usr/libdata/debug//usr/lib/libterminfo.so.2… |
|
1826 Reading symbols from target:/usr/lib/libc.so.12... |
|
1827 Reading /usr/lib/libc.so.12.217.debug from remote target... |
|
1828 Reading /usr/lib/.debug/libc.so.12.217.debug from remote target... |
|
1829 Reading /usr/libdata/debug//usr/lib/libc.so.12.217.debug from remote tar… |
|
1830 Reading /usr/libdata/debug//usr/lib/libc.so.12.217.debug from remote tar… |
|
1831 Reading symbols from target:/usr/libdata/debug//usr/lib/libc.so.12.217.d… |
|
1832 Reading symbols from target:/usr/lib/libgcc_s.so.1... |
|
1833 Reading /usr/lib/libgcc_s.so.1.0.debug from remote target... |
|
1834 Reading /usr/lib/.debug/libgcc_s.so.1.0.debug from remote target... |
|
1835 Reading /usr/libdata/debug//usr/lib/libgcc_s.so.1.0.debug from remote ta… |
|
1836 Reading /usr/libdata/debug//usr/lib/libgcc_s.so.1.0.debug from remote ta… |
|
1837 Reading symbols from target:/usr/libdata/debug//usr/lib/libgcc_s.so.1.0.… |
|
1838 Reading /usr/libexec/ld.elf_so from remote target... |
|
1839 _rtld_debug_state () at /usr/src/libexec/ld.elf_so/rtld.c:1577 |
|
1840 1577 __insn_barrier(); |
|
1841 (gdb) b main |
|
1842 Breakpoint 1 at 0x211c00: file /usr/src/external/mpl/bind/bin/nslookup/.… |
|
1843 (gdb) c |
|
1844 Continuing. |
|
1845 |
|
1846 Breakpoint 1, main (argc=1, argv=0x7f7fffffe768) |
|
1847 at /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslook… |
|
1848 990 main(int argc, char **argv) { |
|
1849 (gdb) bt |
|
1850 #0 main (argc=1, argv=0x7f7fffffe768) |
|
1851 at /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslook… |
|
1852 (gdb) info threads |
|
1853 Id Target Id Frame |
|
1854 * 1 Thread 28353.28353 main (argc=1, argv=0x7f7fffffe768) |
|
1855 at /usr/src/external/mpl/bind/bin/nslookup/../../dist/bin/dig/nslook… |
|
1856 (gdb) b pthread_setname_np |
|
1857 Breakpoint 2 at 0x7f7ff4e0c9e4: file /usr/src/lib/libpthread/pthread.c, … |
|
1858 (gdb) c |
|
1859 Continuing. |
|
1860 [New Thread 28353.27773] |
|
1861 |
|
1862 Thread 1 hit Breakpoint 2, pthread_setname_np (thread=0x7f7ff7e41000, |
|
1863 name=name@entry=0x7f7fffffe610 "work-0", arg=arg@entry=0x0) |
|
1864 at /usr/src/lib/libpthread/pthread.c:792 |
|
1865 792 { |
|
1866 (gdb) info threads |
|
1867 Id Target Id Frame |
|
1868 * 1 Thread 28353.28353 pthread_setname_np (thread=0x7f7ff7e41000, |
|
1869 name=name@entry=0x7f7fffffe610 "work-0", arg=arg@entry=0x0) |
|
1870 at /usr/src/lib/libpthread/pthread.c:792 |
|
1871 2 Thread 28353.27773 0x00007f7ff0aa623a in ___lwp_park60 () from ta… |
|
1872 (gdb) n |
|
1873 796 pthread__error(EINVAL, "Invalid thread", |
|
1874 (gdb) n |
|
1875 799 if (pthread__find(thread) != 0) |
|
1876 (gdb) |
|
1877 802 namelen = snprintf(newname, sizeof(newname), name, arg); |
|
1878 (gdb) |
|
1879 803 if (namelen >= PTHREAD_MAX_NAMELEN_NP) |
|
1880 (gdb) |
|
1881 806 cp = strdup(newname); |
|
1882 (gdb) |
|
1883 807 if (cp == NULL) |
|
1884 (gdb) |
|
1885 810 pthread_mutex_lock(&thread->pt_lock); |
|
1886 (gdb) |
|
1887 811 oldname = thread->pt_name; |
|
1888 (gdb) |
|
1889 812 thread->pt_name = cp; |
|
1890 (gdb) |
|
1891 813 (void)_lwp_setname(thread->pt_lid, cp); |
|
1892 (gdb) |
|
1893 814 pthread_mutex_unlock(&thread->pt_lock); |
|
1894 (gdb) n |
|
1895 816 if (oldname != NULL) |
|
1896 (gdb) n |
|
1897 isc_taskmgr_create (mctx=<optimized out>, workers=workers@entry=1,… |
|
1898 default_quantum@entry=0, nm=nm@entry=0x0, managerp=managerp@entry=0x… |
|
1899 at /usr/src/external/mpl/bind/lib/libisc/../../dist/lib/isc/task.c:1… |
|
1900 1431 for (i = 0; i < workers; i++) { |
|
1901 (gdb) info threads |
|
1902 Id Target Id Frame |
|
1903 * 1 Thread 28353.28353 isc_taskmgr_create (mctx=<optimize… |
|
1904 default_quantum=<optimized out>, default_quantum@entry=0, nm=n… |
|
1905 managerp=managerp@entry=0x418638 <taskmgr>) |
|
1906 at /usr/src/external/mpl/bind/lib/libisc/../../dist/lib/isc/task.c:1… |
|
1907 2 Thread 28353.27773 "work-0" 0x00007f7ff0aa623a in ___lw… |
|
1908 from target:/usr/lib/libc.so.12 |
|
1909 (gdb) dis 1 |
|
1910 (gdb) b exit |
|
1911 Breakpoint 3 at 0x7f7ff0b530e0: exit. (2 locations) |
|
1912 (gdb) c |
|
1913 Continuing. |
|
1914 |
|
1915 Thread 1 hit Breakpoint 2, pthread_setname_np (thread=0x7f7ff7e42c00, |
|
1916 name=name@entry=0x7f7ff5e6324e "isc-timer", arg=arg@entry=… |
|
1917 at /usr/src/lib/libpthread/pthread.c:792 |
|
1918 792 { |
|
1919 (gdb) dis 2 |
|
1920 (gdb) c |
|
1921 Continuing. |
|
1922 Reading /usr/lib/i18n/libUTF8.so.5.0 from remote target... |
|
1923 Reading /usr/lib/i18n/libUTF8.so.5.0.debug from remote target... |
|
1924 Reading /usr/lib/i18n/.debug/libUTF8.so.5.0.debug from remote target... |
|
1925 Reading /usr/libdata/debug//usr/lib/i18n/libUTF8.so.5.0.debug from remot… |
|
1926 Reading /usr/libdata/debug//usr/lib/i18n/libUTF8.so.5.0.debug from remot… |
|
1927 </pre> |
|
1928 <p> |
|
1929 Then, back to the first terminal: |
|
1930 <pre> |
|
1931 > netbsd.org |
|
1932 Server: 62.179.1.62 |
|
1933 Address: 62.179.1.62#53 |
|
1934 |
|
1935 Non-authoritative answer: |
|
1936 Name: netbsd.org |
|
1937 Address: 199.233.217.205 |
|
1938 Name: netbsd.org |
|
1939 Address: 2001:470:a085:999::80 |
|
1940 > exit |
|
1941 |
|
1942 </pre> |
|
1943 <p> |
|
1944 <pre> |
|
1945 Thread 1 hit Breakpoint 3, exit (status=1) at /usr/src/lib/libc/stdlib/e… |
|
1946 55 { |
|
1947 (gdb) info threads |
|
1948 Id Target Id Frame |
|
1949 * 1 Thread 28353.28353 exit (status=1) at /usr/src/lib/libc/stdlib/ex… |
|
1950 (gdb) bt |
|
1951 #0 exit (status=1) at /usr/src/lib/libc/stdlib/exit.c:55 |
|
1952 #1 0x0000000000206122 in ___start () |
|
1953 #2 0x00007f7ff7c0c840 in ?? () from target:/usr/libexec/ld.elf_so |
|
1954 #3 0x0000000000000001 in ?? () |
|
1955 #4 0x00007f7fffffed20 in ?? () |
|
1956 #5 0x0000000000000000 in ?? () |
|
1957 (gdb) kill |
|
1958 Kill the program being debugged? (y or n) y |
|
1959 [Inferior 1 (process 28353) killed] |
|
1960 </pre> |
|
1961 <p> |
|
1962 <b>It worked!</b> |
|
1963 <p> |
|
1964 In order to get this functionality operational I had to implement multip… |
|
1965 post_create_inferior, attach, kill, detach, mourn, join, thread_alive, |
|
1966 resume, wait, fetch_registers, store_registers, read_memory, write_memor… |
|
1967 request_interrupt, supports_read_auxv, read_auxv, |
|
1968 supports_hardware_single_step, sw_breakpoint_from_kind, |
|
1969 supports_z_point_type, insert_point, remove_point, |
|
1970 stopped_by_sw_breakpoint, supports_qxfer_siginfo, qxfer_siginfo, |
|
1971 supports_stopped_by_sw_breakpoint, supports_non_stop, |
|
1972 supports_multi_process, supports_fork_events, supports_vfork_events, |
|
1973 supports_exec_events, supports_disable_randomization, |
|
1974 supports_qxfer_libraries_svr4, qxfer_libraries_svr4, |
|
1975 supports_pid_to_exec_file, pid_to_exec_file, thread_name, |
|
1976 supports_catch_syscall. |
|
1977 <p> |
|
1978 NetBSD is the first BSD and actually the first Open Source UNIX-like OS … |
|
1979 <p> |
|
1980 <h1>Plan for the next milestone</h1> |
|
1981 <p> |
|
1982 Introduce AArch64 support for GDB/NetBSD.</content> |
|
1983 </entry> |
|
1984 <entry> |
|
1985 <id>https://blog.netbsd.org/tnf/entry/gsoc_2020_report_2_fuzzing… |
|
1986 <title type="html">GSoC 2020: Report-2: Fuzzing the NetBSD Netwo… |
|
1987 <author><name>Kamil Rytarowski</name></author> |
|
1988 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
1989 <published>2020-08-30T13:21:57+00:00</published> |
|
1990 <updated>2020-08-30T13:21:57+00:00</updated> |
|
1991 <category term="/The NetBSD Foundation" label="The NetBSD Founda… |
|
1992 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/"… |
|
1993 <category term="rump" scheme="http://roller.apache.org/ns/tags/"… |
|
1994 <category term="fuzzing" scheme="http://roller.apache.org/ns/tag… |
|
1995 <summary type="html">This report was written by Nisarg S. Joshi … |
|
1996 |
|
1997 <p>The objective of this project is to fuzz the various protocols … |
|
1998 <p>You can read the previous post/report <a href="http://b… |
|
1999 <content type="html">This report was written by Nisarg S. Joshi … |
|
2000 |
|
2001 <p>The objective of this project is to fuzz the various protocols … |
|
2002 <p>You can read the previous post/report <a href="http://b… |
|
2003 <p><strong>Overview of the work done:</strong></p&g… |
|
2004 <p>The major time of the phase 1 and 2 were spent in analyzing the… |
|
2005 <ol> |
|
2006 <li>IPv4 (Phase 1)</li> |
|
2007 <li>UDP (Phase 1)</li> |
|
2008 <li>IPv6 (Phase 2)</li> |
|
2009 <li>ICMP (Phase 2)</li> |
|
2010 <li>Ethernet (Phase 2)</li> |
|
2011 </ol> |
|
2012 <p>Quite a good amount of time was spent in understanding the inpu… |
|
2013 <p>For each protocol, mainly 2 things needed to be implemented:&am… |
|
2014 <ol> |
|
2015 <li>The Network Config: the topology for sending and receiving pac… |
|
2016 <li>Packet Creation: Using the information gathered in the code wa… |
|
2017 </ol> |
|
2018 <p>In the next section, a few of the protocols will be explained i… |
|
2019 <p><strong>Protocols</strong></p> |
|
2020 <p>In this section we will talk about the various protocols implem… |
|
2021 <p><strong>IPv4:</strong></p> |
|
2022 <p>IPv4 stands for the Internet protocol version 4. It is one of t… |
|
2023 <p>In order to come up with a strategy for fuzzing, the first step… |
|
2024 <ul> |
|
2025 <li>ip_input() =&gt; Which carries out the processing of a inc… |
|
2026 <li>ip_output() =&gt; Which carries out the processing of an o… |
|
2027 <li>struct ip =&gt; Represents the IP header (src <a href=&… |
|
2028 </ul> |
|
2029 <p>These sections of code represent the working of the input and o… |
|
2030 <p>In order to be able to reach these various aspects of the proto… |
|
2031 <ul> |
|
2032 <li><em>IP Version</em>: Set it to 0x4 which is a 4 bi… |
|
2033 <li><em>IP Header Len</em>: Which is set to a value gr… |
|
2034 <li><em>IP Len</em>: Set it to the size of the random … |
|
2035 <li><em>IP Checksum</em>: We calculate the correct che… |
|
2036 </ul> |
|
2037 <p>Other fields were allowed to be populated randomly by fuzzer in… |
|
2038 <p> |
|
2039 <img src="//netbsd.org/~kamil/gsoc_2020/rumpnetfuzz.png"> |
|
2040 <p>The packet creation code lies in the following section inside [… |
|
2041 <ol> |
|
2042 <li>We call rump_init() to initialize the rumpkernel linked via li… |
|
2043 <li>We setup the Client and server IP addresses</li> |
|
2044 <li>We setup the TUN device by calling the network config function… |
|
2045 <li>We create the packet using the packet creation function utiliz… |
|
2046 <li>Pass this forged packet into the network stack of the rumpkern… |
|
2047 </ol> |
|
2048 <p><strong>IPv6:</strong></p> |
|
2049 <p>IPv6 stands for the Internet protocol version 4. It is the succ… |
|
2050 <p>In order to be able to reach these various aspects of the proto… |
|
2051 <ul> |
|
2052 <li><em>IP Version</em>: Set it to 0x6 which is a 4 bi… |
|
2053 <li><em>IP Hop Limit</em>: This is an alias for TTL. S… |
|
2054 </ul> |
|
2055 <p>Other fields were allowed to be populated randomly by fuzzer in… |
|
2056 <p><strong>UDP:</strong></p> |
|
2057 <p>UDP stands for User Datagram Protocol. It is one of the simples… |
|
2058 <p>Since UDP runs at the transport layer and hence is wrapped up i… |
|
2059 <p>In UDP, we fix the following fields:</p> |
|
2060 <ul> |
|
2061 <li><em>UDP Checksum</em>: Set it to zero in order to … |
|
2062 </ul> |
|
2063 <p><strong>ICMP:</strong></p> |
|
2064 <p>ICMP stands for Internet control message protocol. This protoco… |
|
2065 <ol> |
|
2066 <li>Error messages</li> |
|
2067 <li>Request-Reply Queries.</li> |
|
2068 </ol> |
|
2069 <p>ICMP has a lot of options and is quite generic in the sense tha… |
|
2070 <p>In order to test various ICMP messages and queries, we could no… |
|
2071 <p><strong>Ethernet:</strong></p> |
|
2072 <p>Ethernet protocol defined by the IEEE 802.3 standard is a widel… |
|
2073 <p>In case of Ethernet protocol fuzzing, we had to use a TAP devic… |
|
2074 <p>For packet creation, we set the source and destination MAC addr… |
|
2075 <p><br /><br /></p> |
|
2076 <p><strong>Current Progress and Next steps</strong><… |
|
2077 <p>The project currently has reached a stage where many major inte… |
|
2078 <p>For the next phase of GSoC, the major focus would be to validat… |
|
2079 </entry> |
|
2080 <entry> |
|
2081 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_benchmarking_… |
|
2082 <title type="html">GSoC Reports: Benchmarking NetBSD, second eva… |
|
2083 <author><name>Leonardo Taccari</name></author> |
|
2084 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
2085 <published>2020-08-12T10:09:16+00:00</published> |
|
2086 <updated>2020-08-12T10:09:16+00:00</updated> |
|
2087 <category term="/General" label="General" /> |
|
2088 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/"… |
|
2089 <category term="benchmarking" scheme="http://roller.apache.org/n… |
|
2090 <summary type="html"><p>This report was written by Apurva … |
|
2091 2020.</p> |
|
2092 |
|
2093 <p>This blog post is in continuation of |
|
2094 <a href="//blog.NetBSD.org/tnf/entry/gsoc_reports_benchmarking_n… |
|
2095 blog and describes my progress in the second phase of GSoC 2020 under |
|
2096 The NetBSD Foundation.</p> |
|
2097 |
|
2098 <p>In this phase, I worked on the automation of the regression sui… |
|
2099 using <a href="https://www.phoronix-test-suite.com">Phor… |
|
2100 and its integration with <a href="https://www.gson.org/netbsd/an… |
|
2101 |
|
2102 <p>The automation framework consists of two components Phoromatic … |
|
2103 provided by Phoronix Test Suite in pkgsrc, and Anita, a Python tool for |
|
2104 automating NetBSD installation.</p> |
|
2105 </summary> |
|
2106 <content type="html"><p>This report was written by Apurva … |
|
2107 2020.</p> |
|
2108 |
|
2109 <p>This blog post is in continuation of |
|
2110 <a href="//blog.NetBSD.org/tnf/entry/gsoc_reports_benchmarking_n… |
|
2111 blog and describes my progress in the second phase of GSoC 2020 under |
|
2112 The NetBSD Foundation.</p> |
|
2113 |
|
2114 <p>In this phase, I worked on the automation of the regression sui… |
|
2115 using <a href="https://www.phoronix-test-suite.com">Phor… |
|
2116 and its integration with <a href="https://www.gson.org/netbsd/an… |
|
2117 |
|
2118 <p>The automation framework consists of two components Phoromatic … |
|
2119 provided by Phoronix Test Suite in pkgsrc, and Anita, a Python tool for |
|
2120 automating NetBSD installation.</p> |
|
2121 |
|
2122 <h2>About Phoromatic</h2> |
|
2123 |
|
2124 <p>Phoromatic is a remote management system for the Phoronix Test … |
|
2125 which allows the automatic scheduling of tests, remote installation of |
|
2126 new tests, and the management of multiple test systems through a web |
|
2127 interface. Tests can be scheduled to run on a routine basis across |
|
2128 multiple test systems automatically. Phoromatic can also interface with |
|
2129 revision control systems to offer support for issuing new tests on a |
|
2130 context-basis, such as whenever a Git commit has been pushed. The test |
|
2131 results are then available from the web interface.</p> |
|
2132 |
|
2133 <h3>Phoromatic client-server architecture</h3> |
|
2134 |
|
2135 <p><img src="//www.NetBSD.org/~leot/blog-posts/imgs/pts_ph… |
|
2136 |
|
2137 <p>The Phoromatic server relies upon a PHP/HHVM built-in web server |
|
2138 process and a PTS-hosted WebSocket server. The web server process |
|
2139 handles the web UI and the responsibilities of the Phoromatic server.<… |
|
2140 |
|
2141 <p>Phoromatic clients are testing machines installed with PTS that… |
|
2142 to the Phoromatic web server through the HTTP port of the server.</p&… |
|
2143 |
|
2144 <h3>Phoromatic Setup</h3> |
|
2145 |
|
2146 <p>To start the Phoromatic server, Phoromatic server HTTP port and… |
|
2147 server socket port needs to be set in |
|
2148 <code>~/.phoronix-test-suite/user-config.xml</code> as shown… |
|
2149 |
|
2150 <pre> |
|
2151 ... |
|
2152 &lt;Server&gt; |
|
2153 &lt;RemoteAccessPort&gt;8640&lt;/RemoteAccessPort&… |
|
2154 &lt;Password&gt;&lt;/Password&gt; |
|
2155 &lt;WebSocketPort&gt;8642&lt;/WebSocketPort&gt; |
|
2156 &lt;AdvertiseServiceZeroConf&gt;TRUE&lt;/AdvertiseServ… |
|
2157 &lt;AdvertiseServiceOpenBenchmarkRelay&gt;TRUE&lt;/Adv… |
|
2158 &lt;PhoromaticStorage&gt;~/.phoronix-test-suite/phoromatic… |
|
2159 &lt;/Server&gt; |
|
2160 </pre> |
|
2161 |
|
2162 <h3>Phoromatic Usage</h3> |
|
2163 |
|
2164 <p>To start the Phoromatic web server for controlling local Phoron… |
|
2165 |
|
2166 <p><code>$ phoronix-test-suite start-phoromatic-server</c… |
|
2167 |
|
2168 <p>The Phoromatic web server will be hosted at <code>localho… |
|
2169 |
|
2170 <h4>Phoromatic Clients</h4> |
|
2171 |
|
2172 <p>The Phoromatic client is used for connecting to a Phoromatic se… |
|
2173 |
|
2174 <p>Phoromatic clients can be created and connected to the server u… |
|
2175 |
|
2176 <p><code> |
|
2177 $ phoronix-test-suite phoromatic.connect SERVER_IP:SERVER_HTTP_PORT/ACCO… |
|
2178 </code></p> |
|
2179 |
|
2180 <p>Phoromatic server interacts with the Phoromatic clients through… |
|
2181 |
|
2182 <h4>Phoromatic Test-schedules</h4> |
|
2183 |
|
2184 <p>A test schedule is used to facilitate automatically running a s… |
|
2185 test(s)/suite(s) on either a routine timed basis or whenever triggered |
|
2186 by an external script or process, e.g. Git/VCS commit, manually |
|
2187 triggered, etc. |
|
2188 Phoromatic provides an option for pre-install, pre-run, post-install |
|
2189 and post-run shell scripts that are executed on the Phoromatic |
|
2190 clients. |
|
2191 Test-schedules can be configured to run any tests on any specific |
|
2192 systems.</p> |
|
2193 |
|
2194 <h2>About Anita</h2> |
|
2195 |
|
2196 <p>Anita is a tool for automated testing of the NetBSD operating s… |
|
2197 Using Anita, we can download a NetBSD distribution and install it in a |
|
2198 virtual machine in a fully automated fashion. Anita is written in |
|
2199 Python and uses the pexpect module to &ldquo;screen scrape&rdquo… |
|
2200 output over an emulated serial console and script the installation |
|
2201 procedure.</p> |
|
2202 |
|
2203 <h3>Installation</h3> |
|
2204 |
|
2205 <p>Anita can be installed on NetBSD, Linux and macOS systems using… |
|
2206 |
|
2207 <pre> |
|
2208 $ pip install pexpect |
|
2209 $ git clone https://github.com/gson1703/anita/ |
|
2210 $ python setup.py install |
|
2211 </pre> |
|
2212 |
|
2213 <h2>Phoromatic-Anita Integration</h2> |
|
2214 |
|
2215 <p>I would like to describe the workflow here briefly:</p> |
|
2216 |
|
2217 <ul> |
|
2218 <li>A test-schedule was created on the Phoromatic server meant to … |
|
2219 <code>pts/idle-1.2.0</code> test on the host machine that co… |
|
2220 <a href="https://gist.github.com/apurvanandan1997/48b54402db1df3… |
|
2221 as a pre-run script.</li> |
|
2222 <li>The script performs the following: |
|
2223 |
|
2224 <ul> |
|
2225 <li>Creates a mountable disk image with an executable script for |
|
2226 setting up Phoronix Test Suite and Phoromatic client creation on the |
|
2227 benchmarking VM systems.</li> |
|
2228 <li>Invokes Anita with the appropriate command-line options for |
|
2229 configurations and network setup and mounts the image to run the |
|
2230 configuration script on the VM.</li> |
|
2231 <li>Configuration script performs hostname change, DHCP setup, NFS |
|
2232 setup, <code>PKG_PATH</code> setup, PTS installation, its co… |
|
2233 connecting it to the Phoromatic server through a network bridge.</li&… |
|
2234 </ul> |
|
2235 </li> |
|
2236 <li>Once the benchmarking VM systems get connected to the Phoromat… |
|
2237 server, Phoromatic server identifies the benchmarking VM systems with |
|
2238 their IP address, hostname and MAC address.</li> |
|
2239 <li>After the identification, Phoromatic initiates the pending tes… |
|
2240 VM (test-profiles are downloaded on the go in the VM and executed) and |
|
2241 maintains a history of the test result data.</li> |
|
2242 </ul> |
|
2243 |
|
2244 |
|
2245 <p>Few points to be noted:</p> |
|
2246 |
|
2247 <ul> |
|
2248 <li>I have used a local PKG_PATH with a NFS server setup as PTS 9.… |
|
2249 available in wip and recompiling it would be a wastage of time. Later I |
|
2250 have planned to use the binary shard by Joyent: |
|
2251 <a href="https://pkgsrc.joyent.com/packages/NetBSD/9.99.69/amd64… |
|
2252 updated PTS gets upstreamed.</li> |
|
2253 <li>The host machine needs some one-time manual setup like install… |
|
2254 of QEMU, Anita, pexpect, term, cvs, etc., initial user registration on |
|
2255 Phoromatic server, Phoromatic port setup, network bridge setup. Apart |
|
2256 from this, the rest of the framework does not require user |
|
2257 supervision.</li> |
|
2258 </ul> |
|
2259 |
|
2260 |
|
2261 <h5>VM configuration script</h5> |
|
2262 |
|
2263 <p>The following script is used as a pre-run script in the test-sc… |
|
2264 |
|
2265 <p><a href="https://gist.github.com/apurvanandan1997/48b54… |
|
2266 |
|
2267 <h5>Networking Setup</h5> |
|
2268 |
|
2269 <p>A bridged networking mode configuration of QEMU has been used i… |
|
2270 as multiple VMs will be able to accommodate with a single bridge |
|
2271 (created on the host machine, one-time setup) using |
|
2272 <a href="//man.NetBSD.org/dhcpcd.8">dhcpcd(8)</a>, |
|
2273 without complicated host forwarding setup (Phoromatic server requires |
|
2274 HTTP port forwarding).</p> |
|
2275 |
|
2276 <p>In order to enable bridged networking for your QEMU guests, you… |
|
2277 first create and configure a bridge interface on your host.</p> |
|
2278 |
|
2279 <pre> |
|
2280 # ifconfig virbr0 create |
|
2281 </pre> |
|
2282 |
|
2283 <p>Next, you must specify the newly-created bridge interface in |
|
2284 <code>/etc/qemu/bridge.conf</code>:</p> |
|
2285 |
|
2286 <pre> |
|
2287 $ sudo mkdir /etc/qemu |
|
2288 $ sudo touch /etc/qemu/bridge.conf &amp;&amp; sudo chmod 644 /et… |
|
2289 $ sudo sh -c "echo 'allow virbr0' >> /etc/qemu/brid… |
|
2290 </pre> |
|
2291 |
|
2292 <p>Finally, in order for non-privileged processes to be able to in… |
|
2293 qemu-bridge-helper, you must set the setuid bit on the utility:</p> |
|
2294 |
|
2295 <p><code> |
|
2296 $ sudo chmod u+s /usr/local/libexec/qemu-bridge-helper |
|
2297 </code></p> |
|
2298 |
|
2299 <p>For more details on the bridged mode networking setup in QEMU, … |
|
2300 refer to the following guides:</p> |
|
2301 |
|
2302 <ul> |
|
2303 <li><a href="https://t.pagef.lt/basic-networking-with-qemu… |
|
2304 <li><a href="https://www.NetBSD.org/docs/guide/en/chap-net… |
|
2305 </ul> |
|
2306 |
|
2307 |
|
2308 <h5>Reproducing the framework</h5> |
|
2309 |
|
2310 <p>To reproduce the framework, you need to have Phoronix Test Suit… |
|
2311 Anita, pexpect, cvs, xterm, makefs installed on your host machine.</p… |
|
2312 |
|
2313 <p>For example on NetBSD:</p> |
|
2314 |
|
2315 <pre> |
|
2316 # pkg_add qemu |
|
2317 # pkg_add py37-anita |
|
2318 $ cd pkgsrc/wip/phoronix-test-suite |
|
2319 $ make install |
|
2320 </pre> |
|
2321 |
|
2322 <p>The step-by-step process to get the framework after installing … |
|
2323 including the one-time manual setup, can be summarized as follows: All |
|
2324 control and configuration of the Phoromatic Server is done via the |
|
2325 web-based interface when the Phoromatic Server is active.</p> |
|
2326 |
|
2327 <ul> |
|
2328 <li>Configure the port of Phoromatic server as 8640 and web socket… |
|
2329 8642 as described above.</li> |
|
2330 <li>Start the Phoromatic server using the command stated above. |
|
2331 <img src="//www.NetBSD.org/~leot/blog-posts/imgs/pts_phoromatic-… |
|
2332 <li>Create your user account on the Phoromatic server using the we… |
|
2333 <img src="//www.NetBSD.org/~leot/blog-posts/imgs/pts_phoromatic.… |
|
2334 <li>Disable client system approval for new system addition from the |
|
2335 settings menu in the web interface.</li> |
|
2336 <li>Connect the host machine as a Phoromatic client to the Phoroma… |
|
2337 server using the command stated above.</li> |
|
2338 <li>Create a test-schedule for the host machine with the |
|
2339 <a href="https://gist.github.com/apurvanandan1997/48b54402db1df3… |
|
2340 as specified above and <code>pts/idle-1.2.0</code> as the te… |
|
2341 <img src="//www.NetBSD.org/~leot/blog-posts/imgs/pts_create-sch.… |
|
2342 <li>Execute the test-schedule or assign it on a timed-schedule and… |
|
2343 <img src="//www.NetBSD.org/~leot/blog-posts/imgs/pts_host-machin… |
|
2344 <li>New VM systems with the latest NetBSD-current binaries and pac… |
|
2345 will be created and identified by Phoromatic server automatically.</l… |
|
2346 <li>Once for all, we need to specify what benchmarking test-profil… |
|
2347 to be run on the VM systems in the test-schedules section and it will |
|
2348 be taken care of by Phoromatic. |
|
2349 <img src="//www.NetBSD.org/~leot/blog-posts/imgs/pts_test-sch.pn… |
|
2350 <li>The result history can also be viewed from Phoromatic web inte… |
|
2351 </ul> |
|
2352 |
|
2353 |
|
2354 <p>You can have a look at the video to get a clearer picture of ho… |
|
2355 setup the framework:</p> |
|
2356 |
|
2357 <iframe width="800" height="450" src="https:… |
|
2358 |
|
2359 <h2>Future Plans</h2> |
|
2360 |
|
2361 <p>The regression suite is complete and final tasks of deploying i… |
|
2362 benchmark.NetBSD.org and upstreaming the wip of Phoronix Test Suite |
|
2363 will be done in the final phase of my GSoC project. |
|
2364 I want to thank my mentors for their constant support.</p> |
|
2365 </content> |
|
2366 </entry> |
|
2367 <entry> |
|
2368 <id>https://blog.netbsd.org/tnf/entry/gsoc_2020_second_evaluatio… |
|
2369 <title type="html">GSoC 2020 Second Evaluation Report: Curses Li… |
|
2370 <author><name>Kamil Rytarowski</name></author> |
|
2371 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
2372 <published>2020-08-07T11:21:00+00:00</published> |
|
2373 <updated>2020-08-07T11:47:52+00:00</updated> |
|
2374 <category term="/The NetBSD Foundation" label="The NetBSD Founda… |
|
2375 <category term="curses" scheme="http://roller.apache.org/ns/tags… |
|
2376 <summary type="html"><i>This report was prepared by Naman … |
|
2377 <p>My GSoC project under NetBSD involves the development of test f… |
|
2378 <content type="html"><i>This report was prepared by Naman … |
|
2379 <p>My GSoC project under NetBSD involves the development of test f… |
|
2380 <h2><a id="user-content-complex-characters" class=&qu… |
|
2381 <p>A complex character is a set of associated character, which may… |
|
2382 The <strong>cchar_t</strong> data type represents a complex … |
|
2383 <div class="highlight highlight-source-c"><pre><… |
|
2384 <span class="pl-c1">attr_t</span> attribut… |
|
2385 <span class="pl-k">unsigned</span> element… |
|
2386 <span class="pl-c1">wchar_t</span> vals[CU… |
|
2387 };</pre></div> |
|
2388 <p><em>vals</em> array contains the spacing character … |
|
2389 In this coding period, I wrote tests for routines involving complex char… |
|
2390 <h2><a id="user-content-alternate-character-set" clas… |
|
2391 <p>When you print "BSD", you would send the hex-codes 42… |
|
2392 <div class="highlight highlight-source-shell"><pre>… |
|
2393 <p>to get a lower-right corner glyph. This enables alternate chara… |
|
2394 <div class="highlight highlight-source-shell"><pre>… |
|
2395 <p>These characters are used in <code>box_set()</code>… |
|
2396 <h2><a id="user-content-progress-in-the-second-coding-phas… |
|
2397 <h3><a id="user-content-improvements-in-the-framework"… |
|
2398 <ol> |
|
2399 <li>Added support for testing of functions to be called before <… |
|
2400 <li>Updated the unsupported function definitions with some minor b… |
|
2401 </ol> |
|
2402 <h3><a id="user-content-testing-and-bug-reports" clas… |
|
2403 <ol> |
|
2404 <li>Added tests for following families of functions: |
|
2405 <ul> |
|
2406 <li>Complex character routines.</li> |
|
2407 <li>Line/box drawing routines.</li> |
|
2408 <li>Pad routines.</li> |
|
2409 <li>Window and sub-window operations.</li> |
|
2410 <li>Curson manipulation routines</li> |
|
2411 </ul> |
|
2412 </li> |
|
2413 <li>Reported bugs (and possible fixes if I know): |
|
2414 <ul> |
|
2415 <li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-sin… |
|
2416 <li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-sin… |
|
2417 <li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-sin… |
|
2418 <li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-sin… |
|
2419 <li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-sin… |
|
2420 <li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-sin… |
|
2421 <li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-sin… |
|
2422 </ul> |
|
2423 </li> |
|
2424 </ol> |
|
2425 <p>I would like to thank my mentors Brett and Martin, as well as t… |
|
2426 </entry> |
|
2427 <entry> |
|
2428 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_rumpk… |
|
2429 <title type="html">GSoC Reports: Fuzzing Rumpkernel Syscalls, Pa… |
|
2430 <author><name>Kamil Rytarowski</name></author> |
|
2431 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
2432 <published>2020-08-05T08:42:37+00:00</published> |
|
2433 <updated>2020-08-05T08:42:37+00:00</updated> |
|
2434 <category term="/The NetBSD Foundation" label="The NetBSD Founda… |
|
2435 <category term="fuzzing" scheme="http://roller.apache.org/ns/tag… |
|
2436 <category term="rump" scheme="http://roller.apache.org/ns/tags/"… |
|
2437 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/"… |
|
2438 <summary type="html"><i>This report was prepared by Aditya… |
|
2439 <p>I have been working on Fuzzing Rumpkernel Syscalls. This blogpo… |
|
2440 <content type="html"><i>This report was prepared by Aditya… |
|
2441 <p>I have been working on Fuzzing Rumpkernel Syscalls. This blogpo… |
|
2442 <h2 id="reproducing-crash-found-in-ioctl-">Reproducing c… |
|
2443 <p>Kamil has worked on reproducing the following crash </p> |
|
2444 <pre><code class="lang-bash=">Thread <span clas… |
|
2445 pipe_ioctl (fp=&lt;optimized <span class="hljs-keyword"… |
|
2446 at /usr/src/<span class="hljs-class"><span class=… |
|
2447 <span class="hljs-symbol">warning:</span> Source f… |
|
2448 <span class="hljs-number">1108</span> … |
|
2449 (gdb) bt |
|
2450 <span class="hljs-comment">#0 pipe_ioctl (fp=&lt;op… |
|
2451 at /usr/src/<span class="hljs-class"><span class=… |
|
2452 <span class="hljs-comment">#1 0x000075b0de65083f in sys… |
|
2453 at /usr/src/<span class="hljs-class"><span class=… |
|
2454 <span class="hljs-comment">#2 0x000075b0de6b8957 in sy_… |
|
2455 sy=&lt;optimized <span class="hljs-keyword">out&… |
|
2456 <span class="hljs-comment">#3 sy_invoke (code=54, rval=… |
|
2457 at /usr/src/<span class="hljs-class"><span class=… |
|
2458 <span class="hljs-comment">#4 rump_syscall (num=num@ent… |
|
2459 retval=retval@entry=<span class="hljs-number">0x7f7f… |
|
2460 at /usr/src/<span class="hljs-class"><span class=… |
|
2461 <span class="hljs-comment">#5 0x000075b0de6ad2ca in rum… |
|
2462 data=&lt;optimized <span class="hljs-keyword">ou… |
|
2463 <span class="hljs-comment">#6 0x0000000000400bf7 in mai… |
|
2464 </code></pre> |
|
2465 <p>in the rump using a fuzzer that uses pip2, dup2 and ioctl sysca… |
|
2466 <p><a href="https://github.com/adityavardhanpadala/rumpsys… |
|
2467 <p>Since rump is a multithreaded process. Crash occurs in any of t… |
|
2468 <h2 id="crash-reproducers">Crash Reproducers</h2> |
|
2469 <p>Getting crash reproducers working took quite a while. If we loo… |
|
2470 <pre><code class="lang-cpp=">void HonggfuzzFetchDa… |
|
2471 <span class="hljs-bullet">. |
|
2472 </span><span class="hljs-bullet">. |
|
2473 </span><span class="hljs-bullet">. |
|
2474 </span><span class="hljs-bullet">. |
|
2475 </span><span class="hljs-strong">*buf_ptr = in… |
|
2476 *</span>len<span class="hljs-emphasis">_ptr = … |
|
2477 <span class="hljs-bullet">. |
|
2478 </span><span class="hljs-bullet">. |
|
2479 </span>} |
|
2480 </code></pre> |
|
2481 <p>And if we observe the attribute we notice that <code>inpu… |
|
2482 <pre><code class="lang-cpp="><span class="… |
|
2483 <span class="hljs-keyword">if</span> ((inputFi… |
|
2484 MAP_FAILED) { |
|
2485 PLOG_F(<span class="hljs-string">"mmap(fd=%… |
|
2486 (<span class="hljs-keyword">size_t</span&… |
|
2487 } |
|
2488 </code></pre> |
|
2489 <p>So in a similar approach HF_ITER() can be modified to read inpu… |
|
2490 <p>Attempts have been made to use getchar(3) for fetching the buff… |
|
2491 <p>So we overload HF_ITER() function whenever we require to reprod… |
|
2492 <pre><code class="lang-cpp="> |
|
2493 <span class="hljs-function"><span class="hljs-ke… |
|
2494 <span class="hljs-keyword">void</span> <span cl… |
|
2495 </span>{ |
|
2496 <span class="hljs-meta">#<span class="hljs-meta-… |
|
2497 FILE *fp = fopen(argv[<span class="hljs-number">1<… |
|
2498 data = <span class="hljs-built_in">malloc</span&g… |
|
2499 fread(data, max_size, <span class="hljs-number">1<… |
|
2500 fclose(fp); |
|
2501 <span class="hljs-meta">#<span class="hljs-meta-… |
|
2502 <span class="hljs-comment">// Initialise the rumpker… |
|
2503 <span class="hljs-keyword">if</span>(rump_init… |
|
2504 __builtin_trap(); |
|
2505 } |
|
2506 |
|
2507 <span class="hljs-meta">#<span class="hljs-meta-… |
|
2508 <span class="hljs-function"><span class="hljs-ke… |
|
2509 *buf = (<span class="hljs-keyword">uint8_t</s… |
|
2510 *len = max_size; |
|
2511 <span class="hljs-keyword">return</span>; |
|
2512 } |
|
2513 <span class="hljs-meta">#<span class="hljs-meta-… |
|
2514 <span class="hljs-function">EXTERN <span class="… |
|
2515 <span class="hljs-meta">#<span class="hljs-meta-… |
|
2516 </code></pre> |
|
2517 <p>This way we can easily reproduce crashes that we get and get th… |
|
2518 <h2 id="generating-c-reproducers">Generating C reproduce… |
|
2519 <p>Now the main goal is to create a c file which can reproduce the… |
|
2520 <pre><code class="lang-cpp=">#ifdef CRASH_REPR |
|
2521 FILE *fp = fopen(<span class="hljs-string">"… |
|
2522 fprintf(<span class="hljs-name">fp</span>,… |
|
2523 fclose(<span class="hljs-name">fp</span>)&… |
|
2524 #else |
|
2525 rump_sys_ioctl(<span class="hljs-name">get_u8<… |
|
2526 #endif |
|
2527 </code></pre> |
|
2528 <p>I followed the same above method for all the syscalls that are … |
|
2529 <h3 id="obstacles">Obstacles</h3> |
|
2530 <p>The number of times each syscall is executed before getting to … |
|
2531 <h2 id="to-do">To-Do</h2> |
|
2532 <ul> |
|
2533 <li>./build.sh building rump on linux+netbsd</li> |
|
2534 <li>pregenerating fuzzer input using the implementation similar to… |
|
2535 </ul> |
|
2536 <p>Finally I thank my mentors Siddharth Muralee, Maciej Grochowski… |
|
2537 </entry> |
|
2538 <entry> |
|
2539 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syz… |
|
2540 <title type="html">GSoC Reports: Enhancing Syzkaller support for… |
|
2541 <author><name>Kamil Rytarowski</name></author> |
|
2542 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
2543 <published>2020-08-05T08:10:10+00:00</published> |
|
2544 <updated>2020-08-05T08:10:10+00:00</updated> |
|
2545 <category term="/The NetBSD Foundation" label="The NetBSD Founda… |
|
2546 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/"… |
|
2547 <category term="syzkaller" scheme="http://roller.apache.org/ns/t… |
|
2548 <category term="fuzzing" scheme="http://roller.apache.org/ns/tag… |
|
2549 <summary type="html"><i>This report was prepared by Ayushi… |
|
2550 |
|
2551 <p>As a part of Google summer code 2020, I have been working on &l… |
|
2552 |
|
2553 <p>For work done in the first coding period, you can take a look a… |
|
2554 |
|
2555 <h2>Automation for enhancement</h2> |
|
2556 <p>With an aim of increasing the number of syscalls fuzzed, we hav… |
|
2557 <content type="html"><i>This report was prepared by Ayushi… |
|
2558 |
|
2559 <p>As a part of Google summer code 2020, I have been working on &l… |
|
2560 |
|
2561 <p>For work done in the first coding period, you can take a look a… |
|
2562 |
|
2563 <h2>Automation for enhancement</h2> |
|
2564 <p>With an aim of increasing the number of syscalls fuzzed, we hav… |
|
2565 |
|
2566 <h2>Design</h2> |
|
2567 <p>All the ioctl commands for a device driver in NetBSD are stored… |
|
2568 <ol> |
|
2569 <li>Generating preprocessed files</li> |
|
2570 <li>Extracting information required for generating descriptions<… |
|
2571 <li>Conversion to syzkaller’s grammar</li> |
|
2572 </ol> |
|
2573 |
|
2574 <img src="//netbsd.org/~kamil/gsoc_2020/sys2syz.png" alt=&q… |
|
2575 |
|
2576 |
|
2577 <h2>Generating Preprocessed files</h2> |
|
2578 <p>For a given preprocessed file, c2xml tool outputs the preproces… |
|
2579 |
|
2580 |
|
2581 <h2>Extractor</h2> |
|
2582 <p>Definition of <a href="//man.NetBSD.org/ioctl.9"&g… |
|
2583 |
|
2584 <img src="//netbsd.org/~kamil/gsoc_2020/ioctl_def.png" alt=… |
|
2585 |
|
2586 <p>When we see it from syzkaller’s perspective, there are ba… |
|
2587 |
|
2588 <p>Description of a particular ioctl command acc to syzkaller̵… |
|
2589 |
|
2590 <p>ioctl$FOOIOCTL(fd &ltfd_driver&gt, cmd const[FOOIOCTL],… |
|
2591 |
|
2592 <br> |
|
2593 <img src="//netbsd.org/~kamil/gsoc_2020/ioctl_desc_1.png" a… |
|
2594 <br> |
|
2595 <img src="//netbsd.org/~kamil/gsoc_2020/ioctl_desc_2.png" a… |
|
2596 |
|
2597 <p>These definitions can be grepped from a device’s header f… |
|
2598 |
|
2599 |
|
2600 <h2>To-Do</h2> |
|
2601 <p>The extracted descriptions have to be converted into syzkaller-… |
|
2602 |
|
2603 <h2>Stats</h2> |
|
2604 Along with this, We have continued to add support for few more syscalls … |
|
2605 <ul> |
|
2606 <li>ksem(2) family</li> |
|
2607 <li>mount(2) family</li> |
|
2608 </ul> |
|
2609 Syscalls related to sockets have also been added. This has increased sys… |
|
2610 |
|
2611 <p>Atlast, I would like to thank my mentors - Cryo, Siddharth Mura… |
|
2612 </entry> |
|
2613 <entry> |
|
2614 <id>https://blog.netbsd.org/tnf/entry/the_gnu_gdb_debugger_and2<… |
|
2615 <title type="html">The GNU GDB Debugger and NetBSD (Part 3)</tit… |
|
2616 <author><name>Kamil Rytarowski</name></author> |
|
2617 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
2618 <published>2020-08-04T16:45:37+00:00</published> |
|
2619 <updated>2020-08-04T16:45:37+00:00</updated> |
|
2620 <category term="/The NetBSD Foundation" label="The NetBSD Founda… |
|
2621 <category term="gdb" scheme="http://roller.apache.org/ns/tags/" … |
|
2622 <summary type="html">The NetBSD team of developers maintains two… |
|
2623 <ul> |
|
2624 <li>One in the base-system with a stack of local patches.</li&g… |
|
2625 <li>One in pkgsrc with mostly build fix patches.</li> |
|
2626 </ul> |
|
2627 <p> |
|
2628 The base-system version of GDB (GPLv3) still relies on a set of local pa… |
|
2629 I set a goal to reduce the local patches to bare minimum, ideally reachi… |
|
2630 <content type="html">The NetBSD team of developers maintains two… |
|
2631 <ul> |
|
2632 <li>One in the base-system with a stack of local patches.</li&g… |
|
2633 <li>One in pkgsrc with mostly build fix patches.</li> |
|
2634 </ul> |
|
2635 <p> |
|
2636 The base-system version of GDB (GPLv3) still relies on a set of local pa… |
|
2637 I set a goal to reduce the local patches to bare minimum, ideally reachi… |
|
2638 <p> |
|
2639 <h1>GDB changes</h1> |
|
2640 <p> |
|
2641 I've written an integration of GDB with fork(2) and vfork(2) events. |
|
2642 Unfortunately, this support (present in a local copy of GDB in the base-… |
|
2643 there is a generic kernel regression with the pg_jobc variable. This var… |
|
2644 a reference counter of the number of processes within a process group th… |
|
2645 The semantics of this variable are not very well defined and in the resu… |
|
2646 This unexpected state of pg_jobc resulted in spurious crashes during ker… |
|
2647 new kernel assertions checking for non-negative pg_jobc values were intr… |
|
2648 GDB as a ptrace(2)-based application happened to reproduce negative pg_j… |
|
2649 the further adoption of the fork(2) and vfork(2) patch in GDB, until the… |
|
2650 I was planning to include support for posix_spawn(3) events as well, as … |
|
2651 operation through a syscall, however this is also blocked by the pg_jobc… |
|
2652 <p> |
|
2653 A local patch for GDB is stored |
|
2654 <a href="//netbsd.org/~kamil/patch-00276-gdb-fork-vfork-events.t… |
|
2655 for the time being. |
|
2656 <p> |
|
2657 I've enable multi-process mode in the NetBSD native target. |
|
2658 This enabled proper support for multiple inferiors and ptrace(2) |
|
2659 assisted management of the inferior processes and their threads. |
|
2660 <p> |
|
2661 <pre> |
|
2662 (gdb) info inferior |
|
2663 Num Description Connection Executable |
|
2664 * 1 process 14952 1 (native) /usr/bin/dig |
|
2665 2 &lt;null&gt; 1 (native) |
|
2666 3 process 25684 1 (native) /bin/ls |
|
2667 4 &lt;null&gt; 1 (native) /bin/ls |
|
2668 </pre> |
|
2669 <p> |
|
2670 Without this change, additional inferiors could be already added, but no… |
|
2671 <p> |
|
2672 I've implemented the xfer_partial TARGET_OBJECT_SIGNAL_INFO support… |
|
2673 NetBSD implements reading and overwriting siginfo_t received by the |
|
2674 tracee. With TARGET_OBJECT_SIGNAL_INFO signal information can be |
|
2675 examined and modified through the special variable $_siginfo. |
|
2676 Currently NetBSD uses an identical siginfo type on |
|
2677 all architectures, so there is no support for architecture-specific fiel… |
|
2678 <pre> |
|
2679 (gdb) b main |
|
2680 Breakpoint 1 at 0x71a0 |
|
2681 (gdb) r |
|
2682 Starting program: /bin/ps |
|
2683 |
|
2684 Breakpoint 1, 0x00000000002071a0 in main () |
|
2685 (gdb) p $_siginfo |
|
2686 $1 = {si_pad = {5, 0, 0, 0, 1, 0 <repeats 19 times>, 1, 0 <repe… |
|
2687 _code = 1, _errno = 0, _pad = 0, _reason = {_rt = {_pid = 0, _uid = … |
|
2688 sival_ptr = 0x1}}, _child = {_pid = 0, _uid = 0, _status = 1, … |
|
2689 _fault = {_addr = 0x0, _trap = 1, _trap2 = 0, _trap3 = 0}, _poll =… |
|
2690 _syscall = {_sysnum = 0, _retval = {0, 1}, _error = 0, _args = {0,… |
|
2691 _ptrace_state = {_pe_report_event = 0, _option = {_pe_other_pid = … |
|
2692 </pre> |
|
2693 <p> |
|
2694 NetBSD, contrary to Linux and other BSDs, supports a ptrace(2) operation… |
|
2695 generate a core(5) file from a running process. This operation is used i… |
|
2696 base-system gcore(1) program. The gcore functionality is also delivered … |
|
2697 I have prepared new code for GDB to wire PT_DUMPCORE into the GDB code f… |
|
2698 and thus support GDB's gcore functionality. |
|
2699 This patch is still waiting in upstream review. |
|
2700 A local copy of the patch is |
|
2701 <a href="//netbsd.org/~kamil/patch-00277-gdb-dumpcore.txt"&… |
|
2702 <p> |
|
2703 <pre> |
|
2704 (gdb) r |
|
2705 Starting program: /bin/ps |
|
2706 |
|
2707 Breakpoint 1, 0x00000000002071a0 in main () |
|
2708 (gdb) gcore |
|
2709 Saved corefile core.4378 |
|
2710 (gdb) !file core.4378 |
|
2711 core.4378: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), NetBSD-st… |
|
2712 </pre> |
|
2713 <p> |
|
2714 <h1>Plan for the next milestone</h1> |
|
2715 <p> |
|
2716 Rewrite the gdbserver support and submit upstream. |
|
2717 </content> |
|
2718 </entry> |
|
2719 <entry> |
|
2720 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_benchmarking_… |
|
2721 <title type="html">GSoC Reports: Benchmarking NetBSD, first eval… |
|
2722 <author><name>Leonardo Taccari</name></author> |
|
2723 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
2724 <published>2020-07-16T09:38:58+00:00</published> |
|
2725 <updated>2020-07-17T16:00:11+00:00</updated> |
|
2726 <category term="/General" label="General" /> |
|
2727 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/"… |
|
2728 <category term="benchmarking" scheme="http://roller.apache.org/n… |
|
2729 <category term="pkgsrc" scheme="http://roller.apache.org/ns/tags… |
|
2730 <summary type="html"><p>This report was written by Apurva … |
|
2731 |
|
2732 <p>My GSoC project under NetBSD involves developing an automated |
|
2733 regression and performance test framework for NetBSD that offers |
|
2734 reproducible benchmarking results with detailed history and logs across |
|
2735 various hardware &amp; architectures.</p> |
|
2736 |
|
2737 <p>To achieve this performance testing framework, I am using the |
|
2738 <a href="https://www.phoronix-test-suite.com/">Phoronix … |
|
2739 which is an open-source, cross-platform automated testing/benchmarking |
|
2740 software for Linux, Windows and BSD environments. It allows the |
|
2741 creation of new tests using simple XML files and shell scripts and |
|
2742 integrates with revision control systems for per-commit regression |
|
2743 testing.</p></summary> |
|
2744 <content type="html"><p>This report was written by Apurva … |
|
2745 |
|
2746 <p>My GSoC project under NetBSD involves developing an automated |
|
2747 regression and performance test framework for NetBSD that offers |
|
2748 reproducible benchmarking results with detailed history and logs across |
|
2749 various hardware &amp; architectures.</p> |
|
2750 |
|
2751 <p>To achieve this performance testing framework, I am using the |
|
2752 <a href="https://www.phoronix-test-suite.com/">Phoronix … |
|
2753 which is an open-source, cross-platform automated testing/benchmarking |
|
2754 software for Linux, Windows and BSD environments. It allows the |
|
2755 creation of new tests using simple XML files and shell scripts and |
|
2756 integrates with revision control systems for per-commit regression |
|
2757 testing.</p> |
|
2758 |
|
2759 <h2>About PTS</h2> |
|
2760 |
|
2761 <h3>PTS core</h3> |
|
2762 |
|
2763 <p>PTS core is the engine of the Phoronix Test Suite and handles t… |
|
2764 |
|
2765 <ul> |
|
2766 <li>Regularly updating the test profiles, test suites, and reposit… |
|
2767 <li>Downloading, compilation, installation and evaluation of test … |
|
2768 <li>Test result management and uploading results and user-defined |
|
2769 test-profiles/suites to OpenBenchmarking.org</li> |
|
2770 </ul> |
|
2771 |
|
2772 |
|
2773 <h3>Test-profiles/Suites &amp; OpenBenchmarking</h3> |
|
2774 |
|
2775 <p>Test-profiles are light-weight XMLs and shell scripts that allo… |
|
2776 installation of the benchmarking packages and their evaluation. |
|
2777 Test-profiles generally contains the following files:</p> |
|
2778 |
|
2779 <ul> |
|
2780 <li><code>downloads.xml</code>: Stores the links of th… |
|
2781 required additional packages, patches to be applied, etc. with their |
|
2782 hash sums.</li> |
|
2783 <li><code>install.sh</code>: Actual compilation, insta… |
|
2784 evaluation shell script. Also handles the task of applying patches.</… |
|
2785 <li><code>results-definition.xml</code>: Meta-data to … |
|
2786 test result data (e.g. result data units, LIB or HIB, etc.)</li> |
|
2787 <li><code>test-definition.xml</code>: Meta-data to spe… |
|
2788 such as compatible OS, external dependencies, test arguments, etc.</l… |
|
2789 </ul> |
|
2790 |
|
2791 |
|
2792 <p>A simple test-profile |
|
2793 <a href="https://openbenchmarking.org/innhold/91db3ffff901d12dab… |
|
2794 can be seen to get an idea of the XMLs and shell scripts.</p> |
|
2795 |
|
2796 <p>Test-suites are bundles of related test-profiles or more suites, |
|
2797 focusing on a subsystem or certain category of tests e.g. Disk Test |
|
2798 Suite, Kernel, CPU suite, etc.</p> |
|
2799 |
|
2800 <p><a href="https://openbenchmarking.org/">OpenBen… |
|
2801 repository for storing test profiles, test suites, hence allowing |
|
2802 new/updated tests to be seamlessly obtained via PTS. |
|
2803 OpenBenchmarking.org also provides a platform to store the test result |
|
2804 data openly and do a comparison between any test results stored in the |
|
2805 OpenBenchmarking.org cloud.</p> |
|
2806 |
|
2807 <h2>Usage</h2> |
|
2808 |
|
2809 <h3>Test installation</h3> |
|
2810 |
|
2811 <p>The command:</p> |
|
2812 |
|
2813 <p><code>$ phoronix-test-suite install test-profile-name<… |
|
2814 |
|
2815 <p>Fetches the sources of the tests related to the the test-profil… |
|
2816 <code>~/.phoronix-test-suite/installed-tests</code>, applies… |
|
2817 carries out compilation of test sources for generating the executables |
|
2818 to run the tests.</p> |
|
2819 |
|
2820 <h3>Test execution</h3> |
|
2821 |
|
2822 <p>The command:</p> |
|
2823 |
|
2824 <p><code>$ phoronix-test-suite run test-profile-name</cod… |
|
2825 |
|
2826 <p>Performs installation of the test (similar to <code>$ pho… |
|
2827 install test-profile-name</code>) followed by exectution the binar… |
|
2828 the compiled sources of the test in |
|
2829 <code>~/.phoronix-test-suite/installed-tests</code> and fina… |
|
2830 tests results/outcome to the user and provides an option to upload |
|
2831 results to OpenBenchmarking.org.</p> |
|
2832 |
|
2833 <h2>Progress in the first phase of GSoC</h2> |
|
2834 |
|
2835 <h3>pkgsrc-wip</h3> |
|
2836 |
|
2837 <p>The first task performed by me was upgrading the Phoronix Test … |
|
2838 from version 8.8 to the latest stable version 9.6.1 in <code>pkgsr… |
|
2839 and is available as |
|
2840 <a href="https://pkgsrc.se/wip/phoronix-test-suite">wip/… |
|
2841 You can have a look at the |
|
2842 <a href="https://github.com/phoronix-test-suite/phoronix-test-su… |
|
2843 to know about the improvements between these two versions.</p> |
|
2844 |
|
2845 <h4>Major Commits:</h4> |
|
2846 |
|
2847 <ul> |
|
2848 <li><a href="https://github.com/NetBSD/pkgsrc-wip/commit/0… |
|
2849 <li><a href="https://github.com/NetBSD/pkgsrc-wip/commit/9… |
|
2850 </ul> |
|
2851 |
|
2852 |
|
2853 <p>Currently, the PTS upgrade to 9.6.1 is subject to more modifica… |
|
2854 and fixes before it gets finally merged to the <code>pkgsrc</co… |
|
2855 To test the Phoronix Test Suite 9.6.1 on NetBSD till then, you can |
|
2856 setup <code>pkgsrc-wip</code> on your system via:</p> |
|
2857 |
|
2858 <pre> |
|
2859 $ cd /usr/pkgsrc |
|
2860 $ git clone git://wip.pkgsrc.org/pkgsrc-wip.git wip |
|
2861 </pre> |
|
2862 |
|
2863 <p>To learn more about pkgsrc-wip please see |
|
2864 <a href="https://pkgsrc.org/wip/">The pkgsrc-wip project… |
|
2865 |
|
2866 <p>After setting up <code>pkgsrc-wip</code>, use the f… |
|
2867 installation of PTS 9.6.1:</p> |
|
2868 |
|
2869 <pre> |
|
2870 $ cd /usr/pkgsrc/wip/phoronix-test-suite |
|
2871 $ make install |
|
2872 </pre> |
|
2873 |
|
2874 <p>If any new build/installation errors are encountered, please do… |
|
2875 them in wip/phoronix-test-suite/TODO file and/or contact me.</p> |
|
2876 |
|
2877 <h3>Testing &amp; Debugging</h3> |
|
2878 |
|
2879 <p>As we now know, having a look over OpenBenchmarking.org&rsq… |
|
2880 test-profiles section or using |
|
2881 <code>$ phoronix-test-suite list-available-tests</code>, |
|
2882 there are 309 test-profiles available on PTS for Linux, and only 166 of |
|
2883 309 tests have been ported to NetBSD (can be differentiated by a |
|
2884 thunder symbol on the test profile on the website). These 166 |
|
2885 test-profiles can be fetch, build and executed on NetBSD using just a |
|
2886 single command <code>$ phoronix-test-suite run test-profile-name&l… |
|
2887 ignoring the graphics ones in both Linux &amp; NetBSD as GPU benchma… |
|
2888 wasn&rsquo;t required in the project.</p> |
|
2889 |
|
2890 <p>Many of the test profiles available on NetBSD have installation… |
|
2891 or runtime errors i.e. they don&rsquo;t work out of the box using th… |
|
2892 <code>$ phoronix-test-suite run test-profile-name</code>. I … |
|
2893 test profiles on a NetBSD-current x86_64 QEMU VM (took a lot of time |
|
2894 due to the heavy tests) and have reported the status in the sheet: |
|
2895 <a href="https://drive.google.com/file/d/1gW76JI7W-Jpeczns49k1bB… |
|
2896 |
|
2897 <p>You can have a look at how a PTS test-profile under action look… |
|
2898 |
|
2899 <h4>Aircrack-NG test-profile demonstration</h4> |
|
2900 |
|
2901 <p>Aircrack-ng is a tool for assessing WiFi/WLAN network security.… |
|
2902 |
|
2903 <p>The PTS test-profile is available at: |
|
2904 <a href="https://openbenchmarking.org/test/pts/aircrack-ng"… |
|
2905 |
|
2906 <p><img src="//www.NetBSD.org/~leot/blog-posts/imgs/pts_ai… |
|
2907 |
|
2908 <p>For further information, complete results can be seen at |
|
2909 <a href="https://openbenchmarking.org/result/2007116-NI-AIRCRACK… |
|
2910 |
|
2911 <h4>AOBench test-profile demonstration</h4> |
|
2912 |
|
2913 <p>AOBench is a lightweight ambient occlusion renderer, written in… |
|
2914 |
|
2915 <p>The PTS test-profile is available at: |
|
2916 <a href="https://openbenchmarking.org/test/pts/aircrack-ng"… |
|
2917 |
|
2918 <p><img src="//www.NetBSD.org/~leot/blog-posts/imgs/pts_ao… |
|
2919 |
|
2920 <p>For further information, complete results can be seen at: |
|
2921 <a href="https://openbenchmarking.org/result/2007148-NI-AOBENCHT… |
|
2922 |
|
2923 <h4>Debugging and fixing non-working test-profiles</h4> |
|
2924 |
|
2925 <p>After testing these test-profiles, I debugged the following bui… |
|
2926 errors in the following test-profiles:</p> |
|
2927 |
|
2928 <ul> |
|
2929 <li>pts/t-test1-1.0.1: <code>memalign()</code> not ava… |
|
2930 <li>pts/coremark-1.0.0: calling bmake instead of gmake</li> |
|
2931 <li>pts/apache-siege-1.0.4: Missing <code>signal.h</code&… |
|
2932 <li>pts/mbw-1.0.0: <code>mempcpy()</code> not availabl… |
|
2933 </ul> |
|
2934 |
|
2935 |
|
2936 <p>Fixes to the above bugs can be found in the sheet or in the Git… |
|
2937 repositories shared in the next paragraph.</p> |
|
2938 |
|
2939 <p>The modified test-profiles have been pushed to |
|
2940 <a href="https://github.com/apurvanandan1997/pts-test-profiles-d… |
|
2941 and the fix patches to |
|
2942 <a href="https://github.com/apurvanandan1997/pts-test-profiles-p… |
|
2943 These patches are automatically downloaded by the debugged |
|
2944 test-profiles and automatically applied before building of tests. The |
|
2945 steps to install the debugged test-profiles have been added in the |
|
2946 <code>README.md</code> of |
|
2947 <a href="https://github.com/apurvanandan1997/pts-test-profiles-d… |
|
2948 |
|
2949 <p>These patches have been added in the <code>downloads.xml&… |
|
2950 modified test-profiles, and hence they get fetched during the test |
|
2951 installation. The <code>install.sh</code> script in these mo… |
|
2952 test-profiles applies them on the benchmarking packages if the |
|
2953 operating system is detected as NetBSD, thereby removing the build |
|
2954 errors.</p> |
|
2955 |
|
2956 <h4>coremark test-profile demonstration</h4> |
|
2957 |
|
2958 <p>You can have a look at the patched coremark-1.0.0 test-profile … |
|
2959 execution:</p> |
|
2960 |
|
2961 <p>The patched PTS test-profile is available at: |
|
2962 <a href="https://github.com/apurvanandan1997/pts-test-profiles-d… |
|
2963 |
|
2964 <p>This is a test of EEMBC CoreMark processor benchmark.</p> |
|
2965 |
|
2966 <p><img src="//www.NetBSD.org/~leot/blog-posts/imgs/pts_co… |
|
2967 |
|
2968 <p>For further information, complete results can be seen at: |
|
2969 <a href="https://openbenchmarking.org/result/2007148-NI-LOCALCOR… |
|
2970 |
|
2971 <h4>Porting more test-profiles to NetBSD</h4> |
|
2972 |
|
2973 <p>Attempts were made to port new test-profiles from Linux to NetB… |
|
2974 the major incompatibility issue was missing external dependencies in |
|
2975 NetBSD. Hence, this may reduce the number of test-profiles we can |
|
2976 actually port, but more sincere efforts will be made in this direction |
|
2977 in the next phase.</p> |
|
2978 |
|
2979 <h2>Future Plans</h2> |
|
2980 |
|
2981 <p>My next steps would involve porting the non-available tests to … |
|
2982 i.e. the non-graphics ones available on Linux but unavailable on |
|
2983 NetBSD, and fix the remaining test-profiles for NetBSD.</p> |
|
2984 |
|
2985 <p>After gaining an availability of a decent number of test-profil… |
|
2986 NetBSD, I will use Phoromatic, a remote tests management system for the |
|
2987 PTS (allows the automatic scheduling of tests, remote installation of |
|
2988 new tests, and the management of multiple test systems) to host the |
|
2989 live results of the automated benchmarking framework on |
|
2990 benchmark.NetBSD.org, thereby delivering a functional performance and |
|
2991 regression testing framework.</p> |
|
2992 </content> |
|
2993 </entry> |
|
2994 <entry> |
|
2995 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_enhancing_syz… |
|
2996 <title type="html">GSoC Reports: Enhancing Syzkaller support for… |
|
2997 <author><name>Kamil Rytarowski</name></author> |
|
2998 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
2999 <published>2020-07-13T22:18:21+00:00</published> |
|
3000 <updated>2020-07-13T22:18:21+00:00</updated> |
|
3001 <category term="/The NetBSD Foundation" label="The NetBSD Founda… |
|
3002 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/"… |
|
3003 <summary type="html"><i>This report was prepared by Ayushi… |
|
3004 |
|
3005 <p>I have been working on the project - <a href="https://w… |
|
3006 <content type="html"><i>This report was prepared by Ayushi… |
|
3007 |
|
3008 <p>I have been working on the project - <a href="https://w… |
|
3009 |
|
3010 <h2>Syzkaller</h2> |
|
3011 <p>Syzkaller is a coverage guided fuzzer developed by Google, to f… |
|
3012 |
|
3013 <p> An automated system Syzbot continuously runs the syzkaller fuz… |
|
3014 |
|
3015 <img src="//netbsd.org/~kamil/gsoc_2020/syzbot.png" alt=&qu… |
|
3016 |
|
3017 |
|
3018 <h2>Increasing syscall support</h2> |
|
3019 <p>Initially, the syscall support for <a href="https://gis… |
|
3020 |
|
3021 <p>Major groups of syscalls which have been added: |
|
3022 <ul> |
|
3023 <li>statfs</li> |
|
3024 <li>__getlogin |
|
3025 <li>getsid</li> |
|
3026 <li>mknod</li> |
|
3027 <li>utimes</li> |
|
3028 <li>wait4</li> |
|
3029 <li>seek</li> |
|
3030 <li>setitimer</li> |
|
3031 <li>setpriority</li> |
|
3032 <li>getrusage</li> |
|
3033 <li>clock_settime</li> |
|
3034 <li>nanosleep</li> |
|
3035 <li>getdents </li> |
|
3036 <li>acct </li> |
|
3037 <li>dup</li> |
|
3038 |
|
3039 </ul> |
|
3040 </p> |
|
3041 |
|
3042 <h2>Bugs Found</h2> |
|
3043 |
|
3044 There were a few bugs reported as a result of adding the descriptions fo… |
|
3045 <ul> |
|
3046 <li><a href="https://syzkaller.appspot.com/bug?id=e70e6b07… |
|
3047 <li><a href="https://syzkaller.appspot.com/bug?id=ed011128… |
|
3048 <li><a href="https://syzkaller.appspot.com/bug?id=6d4983b4… |
|
3049 </ul> |
|
3050 |
|
3051 <h2>Stats</h2> |
|
3052 <p>Syscall coverage percent for NetBSD has now increased from near… |
|
3053 <p> Percentage of syscalls covered in few of the other Operating S… |
|
3054 <ul> |
|
3055 <li>Linux: 82</li> |
|
3056 <li>FreeBSD: 37</li> |
|
3057 <li>OpenBSD: 61</li> |
|
3058 </ul> |
|
3059 </p> |
|
3060 <h2>Conclusion</h2> |
|
3061 <p>In the next phase I would be working on generating the syscall … |
|
3062 |
|
3063 <p>Atlast, I would like to thank my mentors Cryo, Siddharth and Sa… |
|
3064 </entry> |
|
3065 <entry> |
|
3066 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_extending_the… |
|
3067 <title type="html">GSoC Reports: Extending the functionality of … |
|
3068 <author><name>Kamil Rytarowski</name></author> |
|
3069 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
3070 <published>2020-07-13T22:05:18+00:00</published> |
|
3071 <updated>2020-07-13T22:20:52+00:00</updated> |
|
3072 <category term="/The NetBSD Foundation" label="The NetBSD Founda… |
|
3073 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/"… |
|
3074 <summary type="html"><i>This report was prepared by Jason … |
|
3075 <p>NetPGP is a library and suite of tools implementing OpenPGP und… |
|
3076 <ol> |
|
3077 <li>Added the Blowfish block cipher</li> |
|
3078 <li>ECDSA key creation</li> |
|
3079 <li>ECDSA signature and verification</li> |
|
3080 <li>Symmetric file encryption/decryption</li> |
|
3081 <li>S2K Iterated+Salt for symmetric encryption</li> |
|
3082 </ol></summary> |
|
3083 <content type="html"><i>This report was prepared by Jason … |
|
3084 <p>NetPGP is a library and suite of tools implementing OpenPGP und… |
|
3085 <ol> |
|
3086 <li>Added the Blowfish block cipher</li> |
|
3087 <li>ECDSA key creation</li> |
|
3088 <li>ECDSA signature and verification</li> |
|
3089 <li>Symmetric file encryption/decryption</li> |
|
3090 <li>S2K Iterated+Salt for symmetric encryption</li> |
|
3091 </ol> |
|
3092 <p>ECDSA key generation is done using the '--ecdsa' flag… |
|
3093 <pre><code>[jhigh@gsoc2020nb gsoc]$ netpgpkeys --generate-ke… |
|
3094 signature secp521r1/ECDSA a0cdb04e3e8c5e34 2020-06-25 |
|
3095 fingerprint d9e0 2ae5 1d2f a9ae eb96 ebd4 a0cd b04e 3e8c 5e34 |
|
3096 uid jhigh@localhost |
|
3097 Enter passphrase for a0cdb04e3e8c5e34: |
|
3098 Repeat passphrase for a0cdb04e3e8c5e34: |
|
3099 generated keys in directory /tmp/a0cdb04e3e8c5e34 |
|
3100 [jhigh@gsoc2020nb gsoc]$ |
|
3101 |
|
3102 [jhigh@gsoc2020nb gsoc]$ ls -l /tmp/a0cdb04e3e8c5e34 |
|
3103 total 16 |
|
3104 -rw------- 1 jhigh wheel 331 Jun 25 16:03 pubring.gpg |
|
3105 -rw------- 1 jhigh wheel 440 Jun 25 16:03 secring.gpg |
|
3106 [jhigh@gsoc2020nb gsoc]$ |
|
3107 </code></pre> |
|
3108 <p>Signing with ECDSA does not require any changes</p> |
|
3109 <pre><code>[jhigh@gsoc2020nb gsoc]$ netpgp --sign --homedir=… |
|
3110 signature secp521r1/ECDSA a0cdb04e3e8c5e34 2020-06-25 |
|
3111 fingerprint d9e0 2ae5 1d2f a9ae eb96 ebd4 a0cd b04e 3e8c 5e34 |
|
3112 uid jhigh@localhost |
|
3113 netpgp passphrase: |
|
3114 [jhigh@gsoc2020nb gsoc]$ |
|
3115 |
|
3116 [jhigh@gsoc2020nb gsoc]$ cat testfile.txt.asc |
|
3117 -----BEGIN PGP MESSAGE----- |
|
3118 |
|
3119 wqcEABMCABYFAl71EYwFAwAAAAAJEKDNsE4+jF40AAAVPgIJASyzuZgyS13FHHF/9qk6E3pY… |
|
3120 tDdkqxYzNIqKnWHaB+a4J+/R7FkZItbC/EyXH5YA68AC1dJ7tRN/tJNIWfYjAgUb75SvM2mL… |
|
3121 qmBo48S0Ai8C82G4nT7/16VF2OOUn7F/3XICghoQOyS1nxJilj8u2uphLOoy9VayL1ErORIZ… |
|
3122 =p30e |
|
3123 -----END PGP MESSAGE----- |
|
3124 [jhigh@gsoc2020nb gsoc]$ |
|
3125 </code></pre> |
|
3126 <p>Verification remains the same, as well.</p> |
|
3127 <pre><code>[jhigh@gsoc2020nb gsoc]$ netpgp --homedir=/tmp/a0… |
|
3128 netpgp: assuming signed data in "testfile.txt" |
|
3129 Good signature for testfile.txt.asc made Thu Jun 25 16:05:16 2020 |
|
3130 using ECDSA key a0cdb04e3e8c5e34 |
|
3131 signature secp521r1/ECDSA a0cdb04e3e8c5e34 2020-06-25 |
|
3132 fingerprint d9e0 2ae5 1d2f a9ae eb96 ebd4 a0cd b04e 3e8c 5e34 |
|
3133 uid jhigh@localhost |
|
3134 [jhigh@gsoc2020nb gsoc]$ |
|
3135 </code></pre> |
|
3136 <p>Symmetric encryption is now possible using the '--symmetri… |
|
3137 <pre><code>[jhigh@gsoc2020nb gsoc]$ netpgp --encrypt --symme… |
|
3138 Enter passphrase: |
|
3139 Repeat passphrase: |
|
3140 [jhigh@gsoc2020nb gsoc]$ |
|
3141 |
|
3142 [jhigh@gsoc2020nb gsoc]$ cat testfile.txt.asc |
|
3143 -----BEGIN PGP MESSAGE----- |
|
3144 |
|
3145 wwwEAwEIc39k1V6xVi3SPwEl2ww75Ufjhw7UA0gO/niahHWK50DFHSD1lB10nUyCTgRLe6iS… |
|
3146 av5Nji9BuQFcrqo03I1jG/L9s/4hww== |
|
3147 =x41O |
|
3148 -----END PGP MESSAGE----- |
|
3149 [jhigh@gsoc2020nb gsoc]$ |
|
3150 </code></pre> |
|
3151 <p>Decryption of symmetric packets requires no changes</p> |
|
3152 <pre><code>[jhigh@gsoc2020nb gsoc]$ netpgp --decrypt testfil… |
|
3153 netpgp passphrase: |
|
3154 [jhigh@gsoc2020nb gsoc]$ |
|
3155 </code></pre> |
|
3156 <p>We added two new flags to support s2k mode 3: '--s2k-mode&… |
|
3157 <pre><code>[jhigh@gsoc2020nb gsoc]$ netpgp --encrypt --symme… |
|
3158 Enter passphrase: |
|
3159 Repeat passphrase: |
|
3160 [jhigh@gsoc2020nb gsoc]$ |
|
3161 |
|
3162 |
|
3163 [jhigh@gsoc2020nb gsoc]$ gpg -d testfile.txt.gpg |
|
3164 gpg: CAST5 encrypted data |
|
3165 gpg: encrypted with 1 passphrase |
|
3166 this |
|
3167 is |
|
3168 a |
|
3169 test |
|
3170 [jhigh@gsoc2020nb gsoc]$ |
|
3171 </code></pre></content> |
|
3172 </entry> |
|
3173 <entry> |
|
3174 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_curses_librar… |
|
3175 <title type="html">GSoC Reports: Curses Library Automated Testin… |
|
3176 <author><name>Kamil Rytarowski</name></author> |
|
3177 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
3178 <published>2020-07-13T21:40:13+00:00</published> |
|
3179 <updated>2020-07-13T22:21:34+00:00</updated> |
|
3180 <category term="/The NetBSD Foundation" label="The NetBSD Founda… |
|
3181 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/"… |
|
3182 <summary type="html"><i>This report was prepared by Naman … |
|
3183 <h2><a id="user-content-introduction" class="anc… |
|
3184 <p>My GSoC project under NetBSD involves the development of test f… |
|
3185 <p>The aim of project is to build a robust test suite for the libr… |
|
3186 <content type="html"><i>This report was prepared by Naman … |
|
3187 <h2><a id="user-content-introduction" class="anc… |
|
3188 <p>My GSoC project under NetBSD involves the development of test f… |
|
3189 <p>The aim of project is to build a robust test suite for the libr… |
|
3190 <h2><a id="user-content-why-did-i-chose-this-project"… |
|
3191 <p>I am a final year undergraduate at Indian Institute of Technolo… |
|
3192 <h2><a id="user-content-test-framwork" class="an… |
|
3193 <p>The testframework consists of 2 programs, director and slave. T… |
|
3194 <p><a target="_blank" rel="noopener noreferrer&q… |
|
3195 <p>The director forks a process operating in pty and executes a sl… |
|
3196 <p>Let's walk through a sample test to understand how this wo… |
|
3197 <pre><code>include start |
|
3198 call win newwin 2 5 2 5 |
|
3199 check win NON_NULL |
|
3200 call OK waddstr $win "Hello World!" |
|
3201 call OK wrefresh $win |
|
3202 compare waddstr_refresh.chk |
|
3203 </code></pre> |
|
3204 <p>This is a basic program which initialises the screen, creates n… |
|
3205 <p>The test file is interpreted by the language parser and the cor… |
|
3206 <p>Along with these, the test framework provides capability to <… |
|
3207 <h2><a id="user-content-progress-till-the-first-evaluation… |
|
3208 <h3><a id="user-content-improvements-in-the-framework"… |
|
3209 <ol> |
|
3210 <li>Automated the checkfile generation that has to be done manuall… |
|
3211 <li>Completed the support for complex chacter tests in director an… |
|
3212 <li>Added features like variable-variable comparison.</li> |
|
3213 <li>Fixed non-critical bugs in the framework.</li> |
|
3214 <li>Refactored the code.</li> |
|
3215 </ol> |
|
3216 <h3><a id="user-content-testing-and-bug-reports" clas… |
|
3217 <ol> |
|
3218 <li>Wrote tests for wide character routines.</li> |
|
3219 <li>Reported bugs (and possible fixes if I know): |
|
3220 <ul> |
|
3221 <li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-sin… |
|
3222 <li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-sin… |
|
3223 <li><a href="https://gnats.netbsd.org/cgi-bin/query-pr-sin… |
|
3224 </ul> |
|
3225 </li> |
|
3226 </ol> |
|
3227 <h2><a id="user-content-project-proposal-and-references&qu… |
|
3228 <ul> |
|
3229 <li>Proposal: <a href="https://github.com/NamanJain8/curse… |
|
3230 <li>Project Repo: <a href="https://github.com/NamanJain8/c… |
|
3231 <li>Test Language Details: <a href="https://github.com/Net… |
|
3232 <li>Wonderful report by Brett Lymn: <a href="https://githu… |
|
3233 </ul></content> |
|
3234 </entry> |
|
3235 <entry> |
|
3236 <id>https://blog.netbsd.org/tnf/entry/gsoc_reports_fuzzing_the_n… |
|
3237 <title type="html">GSoC Reports: Fuzzing the NetBSD Network Stac… |
|
3238 <author><name>Kamil Rytarowski</name></author> |
|
3239 <link rel="alternate" type="text/html" href="https://blog.netbsd… |
|
3240 <published>2020-07-13T14:58:58+00:00</published> |
|
3241 <updated>2020-07-13T22:22:24+00:00</updated> |
|
3242 <category term="/The NetBSD Foundation" label="The NetBSD Founda… |
|
3243 <category term="gsoc" scheme="http://roller.apache.org/ns/tags/"… |
|
3244 <summary type="html"><i>This report was prepared by Nisarg… |
|
3245 <p><strong>Introduction:</strong></p> |
|
3246 <p>The objective of this project is to fuzz the various protocols … |
|
3247 <p><strong>Fuzzing:</strong></p> |
|
3248 <p>Fuzzing or fuzz testing is an automated software testing techni… |
|
3249 <p>There are several tools available today that enable this which … |
|
3250 <p>Fuzzers can be of various types like dumb vs smart, generation-… |
|
3251 <p>Some examples of popular fuzzers are: AFL(American Fuzzy Lop), … |
|
3252 <content type="html"><i>This report was prepared by Nisarg… |
|
3253 <p><strong>Introduction:</strong></p> |
|
3254 <p>The objective of this project is to fuzz the various protocols … |
|
3255 <p><strong>Fuzzing:</strong></p> |
|
3256 <p>Fuzzing or fuzz testing is an automated software testing techni… |
|
3257 <p>There are several tools available today that enable this which … |
|
3258 <p>Fuzzers can be of various types like dumb vs smart, generation-… |
|
3259 <p>Some examples of popular fuzzers are: AFL(American Fuzzy Lop), … |
|
3260 <p><strong>RumpKernel</strong></p> |
|
3261 <p>Kernels can have several different architectures like monolithi… |
|
3262 <p>This idea of rumpkernel is really helpful in fuzzing the compon… |
|
3263 <p><strong>HonggFuzz + Rumpkernel Network Stack:</strong&… |
|
3264 <p>In this project we will use the outlined Rumpkernel&rsquo;s… |
|
3265 <p>The project is hosted on github at: <a href="https://gi… |
|
3266 <p><strong>Our Approach for network stack fuzzing:</stron… |
|
3267 <p>We have planned to fuzz various protocols at different layers o… |
|
3268 <p>The network stack has 2 paths:&nbsp;</p> |
|
3269 <ol> |
|
3270 <li>Input/ingress path</li> |
|
3271 <li>Output/egress path</li> |
|
3272 </ol> |
|
3273 <p>A packet is sent down the network stack via a socket from an ap… |
|
3274 <p>In order to fuzz the output and input path, the network stack s… |
|
3275 <ul> |
|
3276 <li>We have a TUN device to which we can read and write a packet.&… |
|
3277 <li>We have a socket that is bound to the TUN device, which can se… |
|
3278 <li>In order to fuzz the input path, we &ldquo;write&rdquo… |
|
3279 <li>In order to fuzz the output path, we send a packet via the soc… |
|
3280 </ul> |
|
3281 <p>For carrying out all the above setup, we have separated out the… |
|
3282 <p>Also in order to reduce the rejection of packets carrying rando… |
|
3283 <p>With these building blocks we have written programs like hfuzz_… |
|
3284 <p><strong>Current Progress:</strong></p> |
|
3285 <p>Following things were worked upon in the first phase:</p> |
|
3286 <ul> |
|
3287 <li>Getting honggfuzz functional for NetBSD(thanks to Kamil for th… |
|
3288 <li>Coming up with the strategy for network configuration and pack… |
|
3289 <li>Adding fuzzing code for protocols like IP(v4) and UDP.</li&… |
|
3290 <li>Carrying out fuzzing for those protocols.</li> |
|
3291 </ul> |
|
3292 <p><strong>Next Steps:</strong></p> |
|
3293 <p>As next steps following things are planned for upcoming phase:&… |
|
3294 <ul> |
|
3295 <li>Making changes and improvements by taking suggestions from the… |
|
3296 <li>Adding support for ICMP, IP(v6), TCP and later on for Ethernet… |
|
3297 <li>Analyze and come up with effective ways to improve the fuzzing… |
|
3298 <li>Standardize the code to be extensible for adding future protoc… |
|
3299 </ul></content> |
|
3300 </entry> |
|
3301 </feed> |
|
3302 |
