Zola2025-12-18T00:00:00+00:00https://jackbondpreston.me/atom.xmlunique hanzi extractor2025-12-18T00:00:00+00:002025-12-18T00:00:00+00:00
Unknown
https://jackbondpreston.me/unique-hanzi/<style>
div#output-box, textarea#input-box {
width: 100%;
min-height: 30vh;
padding: 10px;
font-size: 1rem;
}
div#output-box {
border: 2px solid rgb(252, 169, 184);
}
</style>
<script src="/js/hanzi.js" defer></script>
<p>when learning Mandarin Chinese, one may want to extract the unique characters from a block of text. for example, one could take an article and create worksheets using the <a href="https://mandarinminutes.com/worksheets">Mandarin Minutes worksheet tool</a> in order to practice writing all the unique characters used within said text.</p>
<p>the tool below allows this - input the text into the top box, and the bottom box will contain the list of unique characters. the tool makes a best effort to filter out non-useful characters like punctuation and spacing. the tool should also work with other languages. for non-logographical languages it should be largely uninteresting; for any sufficiently large text the output will just consist of the whole alphabet in lower and upper case.</p>
<h2 id="input">input<a href="#input" aria-label="Anchor link for: input" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<textarea id="input-box"></textarea>
<h2 id="output">output<a href="#output" aria-label="Anchor link for: output" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<div id="output-box"></div>DPDK Summit 2024: OpenSSL Crypto PMD - Analysis and Optimisations2024-10-23T00:00:00+00:002024-10-23T00:00:00+00:00
Unknown
https://jackbondpreston.me/dpdk-summit-2024/<h2 id="summary">summary<a href="#summary" aria-label="Anchor link for: summary" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p>this year I had the opportunity to present my recent work on <a href="https://www.dpdk.org/">DPDK</a>. the work consists of performance analysis and optimisations to the OpenSSL crypto poll mode driver (PMD). I identified several major performance flaws with the implementation, and integration with OpenSSL, and upstreamed changes to address these. the talk goes through the details of the findings, performance impacts of the changes, and discussions around future work.</p>
<h2 id="recording">recording<a href="#recording" aria-label="Anchor link for: recording" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<iframe width="560" height="315" src="https://www.youtube.com/embed/sPhktvMBYhs?si=UInBMrAUxs2R_raf" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe>
<h2 id="slides">slides<a href="#slides" aria-label="Anchor link for: slides" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p><a href="/pdf/dpdksummit2024.pdf">pdf of the slides used for the talk</a></p>
Linux syscall hooking2024-09-26T00:00:00+00:002024-09-26T00:00:00+00:00
Unknown
https://jackbondpreston.me/syscalls/<p>sometimes, you might want to hook Linux syscalls from a binary. it's probably not for a very elegant reason, but I'm not here to judge. in this article we discuss a few options for doing this. these will be focussed on x86 platforms running Linux, but many of these techniques should also be possible on other ISAs, with appropriate changes.</p>
<h2 id="binary-rewriting">binary rewriting<a href="#binary-rewriting" aria-label="Anchor link for: binary-rewriting" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p>one way to approach the problem is simply to rewrite binaries at runtime. one can set executable text pages to be writeable, swap out some instructions, and then set them back to be read-only and executable. how can this look in practice?</p>
<p>one has to consider the limitations of binary rewriting in this manner. we can only replace syscall instructions with instructions of the same length. thus, we have limited flexibility in how to implement a syscall hooking function. absolute calls to some custom handler address would require further code modification than is possible, in order to prepare a register with the jump address. we can do relative jumps, but we need to find some space within the maximum range to put our handler function.</p>
<h3 id="zpoline">zpoline</h3>
<p>zpoline<sup><a href="#ref1" style="text-decoration: none;">[1]</a></sup> is one approach to get around these limitations, on x86 platforms. zpoline replaces all <code>syscall</code>/<code>sysenter</code> instructions with <code>call *%rax</code>. this takes advantage of two things:</p>
<ol>
<li><code>rax</code> will always contain the target syscall number, thus holding a bounded low number. this allows us to rewrite the virtual memory in the range <code>[0x00, (SYSCALL_MAX + n)]</code> to have some custom handler code of length <code>n</code>.</li>
<li><code>call *%rax</code> is the same size as <code>syscall</code> (2 bytes)<sup><a href="#ref2" style="text-decoration: none;">[2]</a></sup>.
this means we can <code>mmap(0)</code> at the start of runtime, and create fall-through nops, followed by a jump to a syscall hook function. the performance of this should be pretty good, not much different to adding an extra function call to each syscall, which is all we want to do anyway.</li>
</ol>
<p>okay, but is it ok to <code>mmap()</code> the zero virtual address? well, not really. for starters, you probably can't do it as a non-privileged user. Linux has a setting <code>mmap_min_addr</code>, which disallows any <code>mmap()</code>s below a given address<sup><a href="#ref3" style="text-decoration: none;">[3]</a></sup>. secondly, this can cause certain null-pointer bugs to exhibit weird behaviour, instead of throwing segmentation faults.</p>
<p>thus, this option does work, and may be suitable for certain usecases, but it does have large flaws (like all of the options).</p>
<h3 id="custom-trampolines">custom trampolines</h3>
<p>an alternative that doesn't involve <code>mmap()</code>ing the zero page is used by AMD's Onload<sup><a href="#ref4" style="text-decoration: none;">[4]</a></sup> - try to replace</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>b8 NN NN NN NN mov eax,0xN ; N = syscall number
</span><span>0f 05 syscall
</span></code></pre>
<p>with</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>b8 TT TT TT TT mov eax,0xTTTTTTTT ; 0xTTTTTTTT = trampoline func address
</span><span>ff d0 call rax
</span></code></pre>
<p>where <code>0xTTTTTTTT</code> is the address of some trampoline function we have crafted. we can get an address to put this function at by using <code>mmap(..., MMAP_32BIT)</code>.</p>
<p>another option here is to try to get some virtual memory within a certain distance from the syscall code, and use relative call instructions.</p>
<p>note that this method is not as reliable. it relies on being able to find syscall-performing code which can be modified in this way, which is not guaranteed. thus, it could fail in some situations. it does circumvent the requirement for an ugly <code>mmap(0)</code>, however.</p>
<h2 id="syscall-user-dispatch-sud">syscall user dispatch (SUD)<a href="#syscall-user-dispatch-sud" aria-label="Anchor link for: syscall-user-dispatch-sud" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p>SUD<sup><a href="#ref5" style="text-decoration: none;">[5]</a></sup> is probably the most "correct" way to go about doing this. this kernel feature was initially created for Wine. SUD allows the userspace program to request the kernel traps all syscalls, raising a <code>SIGSYS</code> signal every time. a certain range of program memory can be excluded from this trapping, and a flag can be used to toggle trapping off or on at runtime (although the excluded range will never trap, regardless of this flag).</p>
<p>this is is enabled via <code>prctl()</code>:</p>
<pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">prctl</span><span>(</span><span style="color:#bf616a;">PR_SET_SYSCALL_USER_DISPATCH</span><span>, </span><span style="color:#bf616a;">PR_SYS_DISPATCH_ON</span><span>, </span><span style="color:#b48ead;">unsigned long </span><span style="color:#bf616a;">off</span><span>, </span><span style="color:#b48ead;">unsigned long </span><span style="color:#bf616a;">size</span><span>, int8_t *</span><span style="color:#bf616a;">switch</span><span>);
</span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">prctl</span><span>(</span><span style="color:#bf616a;">PR_SET_SYSCALL_USER_DISPATCH</span><span>, </span><span style="color:#bf616a;">PR_SYS_DISPATCH_OFF</span><span>, </span><span style="color:#d08770;">0</span><span style="color:#b48ead;">L</span><span>, </span><span style="color:#d08770;">0</span><span style="color:#b48ead;">L</span><span>, </span><span style="color:#d08770;">0</span><span style="color:#b48ead;">L</span><span>);
</span></code></pre>
<p>where <code>off</code> is the start of the non-trapping memory region, and <code>size</code> is it's length. <code>switch</code> is a pointer to the aforementioned toggleable flag.</p>
<p>here is a very basic example of using SUD:</p>
<pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">#define </span><span>_GNU_SOURCE
</span><span>
</span><span style="color:#b48ead;">#include </span><span><</span><span style="color:#a3be8c;">signal.h</span><span>>
</span><span style="color:#b48ead;">#include </span><span><</span><span style="color:#a3be8c;">stdbool.h</span><span>>
</span><span style="color:#b48ead;">#include </span><span><</span><span style="color:#a3be8c;">stddef.h</span><span>>
</span><span style="color:#b48ead;">#include </span><span><</span><span style="color:#a3be8c;">stdio.h</span><span>>
</span><span style="color:#b48ead;">#include </span><span><</span><span style="color:#a3be8c;">stdlib.h</span><span>>
</span><span style="color:#b48ead;">#include </span><span><</span><span style="color:#a3be8c;">string.h</span><span>>
</span><span style="color:#b48ead;">#include </span><span><</span><span style="color:#a3be8c;">sys/prctl.h</span><span>>
</span><span style="color:#b48ead;">#include </span><span><</span><span style="color:#a3be8c;">sys/syscall.h</span><span>>
</span><span style="color:#b48ead;">#include </span><span><</span><span style="color:#a3be8c;">sys/types.h</span><span>>
</span><span>
</span><span style="color:#b48ead;">bool</span><span> selector = SYSCALL_DISPATCH_FILTER_ALLOW;
</span><span>
</span><span style="color:#b48ead;">static volatile void </span><span style="color:#8fa1b3;">handle_sigsys</span><span>(</span><span style="color:#b48ead;">int </span><span style="color:#bf616a;">sig</span><span>, siginfo_t *</span><span style="color:#bf616a;">info</span><span>, </span><span style="color:#b48ead;">void </span><span>*</span><span style="color:#bf616a;">ucontext</span><span>) {
</span><span> </span><span style="color:#b48ead;">char</span><span> buf[</span><span style="color:#d08770;">128</span><span>];
</span><span> </span><span style="color:#b48ead;">int</span><span> len;
</span><span> </span><span style="color:#b48ead;">int</span><span> ret;
</span><span>
</span><span> len = </span><span style="color:#96b5b4;">snprintf</span><span>(buf, </span><span style="color:#d08770;">1024</span><span>, "</span><span style="color:#a3be8c;">Caught syscall with number 0x</span><span style="color:#d08770;">%x</span><span style="color:#96b5b4;">\n</span><span>", info->si_syscall);
</span><span> selector = SYSCALL_DISPATCH_FILTER_ALLOW;
</span><span> </span><span style="color:#bf616a;">write</span><span>(</span><span style="color:#d08770;">1</span><span>, buf, len);
</span><span>}
</span><span>
</span><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>(</span><span style="color:#b48ead;">void</span><span>) {
</span><span> </span><span style="color:#b48ead;">int</span><span> err;
</span><span>
</span><span> </span><span style="color:#65737e;">/* install SIGSYS signal handler */
</span><span> </span><span style="color:#b48ead;">struct</span><span> sigaction act;
</span><span> </span><span style="color:#96b5b4;">memset</span><span>(&act, </span><span style="color:#d08770;">0</span><span>, sizeof(act));
</span><span>
</span><span> sigset_t mask;
</span><span> </span><span style="color:#bf616a;">sigemptyset</span><span>(&mask);
</span><span>
</span><span> act.</span><span style="color:#bf616a;">sa_sigaction </span><span>= handle_sigsys;
</span><span> act.</span><span style="color:#bf616a;">sa_flags </span><span>= SA_SIGINFO;
</span><span> act.</span><span style="color:#bf616a;">sa_mask </span><span>= mask;
</span><span>
</span><span> </span><span style="color:#b48ead;">if </span><span>(err = </span><span style="color:#bf616a;">sigaction</span><span>(SIGSYS, &act, </span><span style="color:#d08770;">NULL</span><span>)) {
</span><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">sigaction failed: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\n</span><span>", err);
</span><span> </span><span style="color:#96b5b4;">exit</span><span>(-</span><span style="color:#d08770;">1</span><span>);
</span><span> }
</span><span>
</span><span> </span><span style="color:#65737e;">/* enable SUD */
</span><span> </span><span style="color:#b48ead;">if </span><span>(err = </span><span style="color:#bf616a;">prctl</span><span>(PR_SET_SYSCALL_USER_DISPATCH, PR_SYS_DISPATCH_ON, </span><span style="color:#d08770;">0</span><span>, </span><span style="color:#d08770;">0</span><span>, &selector)) {
</span><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">prctl failed: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\n</span><span>", err);
</span><span> </span><span style="color:#96b5b4;">exit</span><span>(-</span><span style="color:#d08770;">1</span><span>);
</span><span> }
</span><span>
</span><span> selector = SYSCALL_DISPATCH_FILTER_BLOCK;
</span><span> </span><span style="color:#bf616a;">syscall</span><span>(SYS_write);
</span><span>
</span><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>;
</span><span>}
</span></code></pre>
<p>running this will produce the following:</p>
<pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Caught syscall with number 0x38
</span></code></pre>
<p>where syscall number <code>0x38</code> is <code>SYS_write</code> (on my system).</p>
<blockquote class="note">
<div class="icon">
<?xml version="1.0" ?><svg height="48" viewBox="0 0 48 48" width="48" xmlns="http://www.w3.org/2000/svg"><path d="M22 34h4v-12h-4v12zm2-30c-11.05 0-20 8.95-20 20s8.95 20 20 20 20-8.95 20-20-8.95-20-20-20zm0 36c-8.82 0-16-7.18-16-16s7.18-16 16-16 16 7.18 16 16-7.18 16-16 16zm-2-22h4v-4h-4v4z" fill="currentColor"/></svg>
</div>
<div class="content">
<p><strong>note</strong></p>
<p>we've had to disable syscall hooking within <code>handle_sigsys()</code>, before we perform the <code>write</code> syscall. this could also be achieved using the offset exclusion mechanism, with a caveat: SUD also needs to be disabled at signal handler return. this is because, depending on some factors, the signal handler will return to a signal trampoline, which exists somewhere else in memory (VDSO or libc). this trampoline will then perform the <code>sigreturn</code> syscall, which - if you had only excluded the handler function memory itself - would cause us to raise a <code>SIGSYS</code> inside a <code>SIGSYS</code> handler (oops).</p>
</div>
</blockquote>
<p>SUD should have pretty good performance, although it is still switching from user space to kernel space (on syscall), and back again (on SIGSYS handling) - unlike the binary rewriting methods.</p>
<h2 id="rewriting-the-kernel-syscall-table">rewriting the kernel syscall table<a href="#rewriting-the-kernel-syscall-table" aria-label="Anchor link for: rewriting-the-kernel-syscall-table" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p>Linux kernel modules could also try to patch the kernel's syscall table.</p>
<p>it used to be the case that if Linux is configured with <code>CONFIG_KALLSYMS_ALL=y</code>:</p>
<pre data-lang="shell" style="background-color:#2b303b;color:#c0c5ce;" class="language-shell "><code class="language-shell" data-lang="shell"><span>$ zgrep CONFIG_KALLSYMS_ALL /proc/config.gz
</span><span>CONFIG_KALLSYMS_ALL=y
</span></code></pre>
<p>then we can find the address of the syscall table in <code>kallsyms</code>:</p>
<pre data-lang="shell" style="background-color:#2b303b;color:#c0c5ce;" class="language-shell "><code class="language-shell" data-lang="shell"><span>$ sudo grep sys_call_table /proc/kallsyms
</span><span>ffffffff854017e0 D sys_call_table
</span></code></pre>
<p>and thus programmatically:</p>
<pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">void </span><span>*sys_call_table = </span><span style="color:#bf616a;">kallsyms_lookup_name</span><span>("</span><span style="color:#a3be8c;">sys_call_table</span><span>");
</span></code></pre>
<p>once the syscall table address has been obtained, the handlers for specific syscall(s) could then be overwritten, in order to hook them.
however, this kind of symbol lookup was generally considered very naughty, and <code>kallsyms_lookup_name()</code> was unexported in Linux 5.7<sup><a href="#ref6" style="text-decoration: none;">[6]</a></sup>. there should be some feasible alternatives to do the same thing, but they would be slower, uglier, and more painful.</p>
<h2 id="modifying-ia32-lstar-msr">modifying IA32_LSTAR MSR<a href="#modifying-ia32-lstar-msr" aria-label="Anchor link for: modifying-ia32-lstar-msr" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p>another method which has been used successfully<sup><a href="#ref7" style="text-decoration: none;">[7]</a></sup> involves rewriting the x86 MSR <code>IA32_LSTAR</code>. this register holds the destination address which the CPU will load into <code>RIP</code> when taking a syscall<sup><a href="#ref2" style="text-decoration: none;">[2]</a></sup>. one can write a kernel module which modifies this register to contain a custom syscall handler address. this syscall handler function can hook syscalls before dispatching as normal (or not).</p>
<p>now that <code>kallsysms_lookup_name()</code> is no longer exported, this method also becomes problematic. as the example kernel module in <a href="#ref7" style="text-decoration: none;">[7]</a> demonstrates, we still need to be able to grab some kernel symbol addresses in order to correctly implement the dispatcher.</p>
<p>however, this does propose another idea for implementing the previous method: using <code>IA32_LSTAR</code> to ascertain the address of the Linux syscall handler, instead of kallsyms. indeed, Onload has also exploited this method as a workaround for the unexporting of <code>kallsyms_lookup_name()</code><sup><a href="#ref8" style="text-decoration: none;">[8]</a></sup>.</p>
<h2 id="citations">citations<a href="#citations" aria-label="Anchor link for: citations" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p id="ref1">[1] <a href="https://www.usenix.org/conference/atc23/presentation/yasukata">https://www.usenix.org/conference/atc23/presentation/yasukata</a> <a href="javascript:history.back()" style="text-decoration: none;">(↑)</a></p><p id="ref2">[2] <a href="https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdm.html">https://www.intel.com/content/www/us/en/developer/articles/technical/intel-sdm.html</a> <a href="javascript:history.back()" style="text-decoration: none;">(↑)</a></p><p id="ref3">[3] <a href="https://elixir.bootlin.com/linux/v6.11/source/arch/s390/mm/mmap.c#L137">https://elixir.bootlin.com/linux/v6.11/source/arch/s390/mm/mmap.c#L137</a> <a href="javascript:history.back()" style="text-decoration: none;">(↑)</a></p><p id="ref4">[4] <a href="https://github.com/Xilinx-CNS/onload/blob/d5984bf52fcfba0cd75df8a1f8828f3363f6e164/src/lib/transport/unix/fdtable.c#L311">https://github.com/Xilinx-CNS/onload/blob/d5984bf52fcfba0cd75df8a1f8828f3363f6e164/src/lib/transport/unix/fdtable.c#L311</a> <a href="javascript:history.back()" style="text-decoration: none;">(↑)</a></p><p id="ref5">[5] <a href="https://docs.kernel.org/admin-guide/syscall-user-dispatch.html">https://docs.kernel.org/admin-guide/syscall-user-dispatch.html</a> <a href="javascript:history.back()" style="text-decoration: none;">(↑)</a></p><p id="ref6">[6] <a href="https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0bd476e6c67190b5eb7b6e105c8db8ff61103281">https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0bd476e6c67190b5eb7b6e105c8db8ff61103281</a> <a href="javascript:history.back()" style="text-decoration: none;">(↑)</a></p><p id="ref7">[7] <a href="https://vvdveen.com/data/lstar.txt">https://vvdveen.com/data/lstar.txt</a> <a href="javascript:history.back()" style="text-decoration: none;">(↑)</a></p><p id="ref8">[8] <a href="https://github.com/Xilinx-CNS/onload/blob/083e5959ef76632ae3cd3d4356fb079ce0c570d1/src/driver/linux_resource/syscall_x86.c#L347">https://github.com/Xilinx-CNS/onload/blob/083e5959ef76632ae3cd3d4356fb079ce0c570d1/src/driver/linux_resource/syscall_x86.c#L347</a> <a href="javascript:history.back()" style="text-decoration: none;">(↑)</a></p>学中文学了一年了2024-09-23T00:00:00+00:002024-09-23T00:00:00+00:00
Unknown
https://jackbondpreston.me/xue-zhong-wen-xue-liao-yi-nian-liao/<p>大约一年前，我开始认真地学习中文。因此，我觉得我应该写一下我的经历。</p>
<p>我刚开始学习时，我只用了一个方法：<a href="https://apps.ankiweb.net/">Anki</a>卡片。我也认为在语言学习方法上，Anki是最有效的。但是为了提高听力和口语，它不太好。</p>
<p>这就是为什么去年我也开始在Italki上上课。因为<a href="https://www.italki.com/">Italki</a>有很多的中文老师，所以你可以选择一个最适合你的学习方式的。对我来说，这是最有效方法提高我的口语。</p>
<p>因为在今年上半年的时候，我还感觉我的听力水平不好，所以我找了方法来提高。我发现了大部分人建议听博客，然后我找到一些有用的博客。我最喜欢的是这些：</p>
<ol>
<li><a href="https://open.spotify.com/show/04re9FvL1xviHGWvXKoAhZ?si=d7bc25d6f481427b">听故事说中文</a></li>
<li><a href="https://open.spotify.com/show/7sHdRAysnMygox20EuYlwp?si=6ec8583891484aef">猫咪中文</a></li>
<li><a href="https://open.spotify.com/show/6mJNegfDGmNaG1mWJtZJed?si=15f5851993d74a48">茶歇中文</a></li>
</ol>
<p>未来将要发生什么？我还不知道，但是我希望我可以继续进步。明年三月我打算和我的对象一起去中国旅游。其实我十分期待这个旅行，因为我不但没去过中国，甚至连亚洲也没去过！既然中国菜非常好吃，我应该学习怎么流利地点菜。我也希望在中国我可以够有信心尝试和中国人聊聊天儿。</p>
CHERI2022-11-19T00:00:00+00:002022-11-19T00:00:00+00:00
Unknown
https://jackbondpreston.me/cheri/<h2 id="preamble">preamble<a href="#preamble" aria-label="Anchor link for: preamble" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p><a href="https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/">CHERI</a> is an acronym for Capability Hardware Enhanced RISC Instructions. it is a security-focussed project aimed at improving memory protection at the hardware level. the project is complex and it has many potential applications.</p>
<p>in this article I will go into some basics to give an understanding behind some changes that CHERI makes to how programs execute and are written. this will be focussed almost entirely in C, as this is where my experience lies - it is also where some of the effects of CHERI are most easily felt.this article is going to be a <em>very simplistic</em> introduction to CHERI, and I'm going to attempt to explain the basics behind everything I cover. a basic understanding of C will be necessary.</p>
<p><em><strong>note:</strong></em> <a href="https://www.arm.com/architecture/cpu/morello">the Morello platform</a> is an evaluation board produced by <a href="https://www.arm.com/">Arm</a> to provide a physical implementation of CHERI extending <a href="https://en.wikipedia.org/wiki/AArch64">the Arm AArch64 ISA</a>. I previously worked on this platform at Arm, <a href="https://git.morello-project.org/morello/musl-libc/">porting the musl C library to Morello</a>. implementations for CHERI that are worth looking into from a more open perspective <a href="https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-951.pdf"> are the MIPS (chapter 4) and RISC-V (chapter 5) ones</a>. Morello is the only implementation that exists in a true hard core format, afaik - but this is obviously hard to obtain so you'll just be playing around with emulators/models anyway.</p>
<h2 id="memory-safety-bugs">memory safety bugs<a href="#memory-safety-bugs" aria-label="Anchor link for: memory-safety-bugs" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p>to understand how CHERI tries to fix some simple issues, we'll first look at some simplified examples of issues that arise when we aren't using a CHERI-based architecture.</p>
<h3 id="a-simple-memory-safety-bug">a simple memory safety bug</h3>
<p>let's take a look at this C code:</p>
<pre data-linenos data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><table><tbody><tr><td>1</td><td><span style="color:#b48ead;">#include </span><span><</span><span style="color:#a3be8c;">stdio.h</span><span>>
</span></td></tr><tr><td>2</td><td><span>
</span></td></tr><tr><td>3</td><td><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>() {
</span></td></tr><tr><td>4</td><td><span> </span><span style="color:#b48ead;">char</span><span> my_perfect_string[] = "</span><span style="color:#a3be8c;">what a beautiful string</span><span>"; </span><span style="color:#65737e;">// so beautiful, I sure hope no-one touches it
</span></td></tr><tr><td>5</td><td><span> </span><span style="color:#b48ead;">char</span><span> user_name[</span><span style="color:#d08770;">32</span><span>];
</span></td></tr><tr><td>6</td><td><span>
</span></td></tr><tr><td>7</td><td><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">enter your name: </span><span>");
</span></td></tr><tr><td>8</td><td><span> </span><span style="color:#96b5b4;">fgets</span><span>(user_name, </span><span style="color:#d08770;">1000</span><span>, stdin); </span><span style="color:#65737e;">// get user's name from stdin
</span></td></tr><tr><td>9</td><td><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">hello </span><span style="color:#d08770;">%s</span><span>", user_name);
</span></td></tr><tr><td>10</td><td><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">my_perfect_string: </span><span style="color:#d08770;">%s</span><span style="color:#96b5b4;">\n</span><span>", my_perfect_string);
</span></td></tr><tr><td>11</td><td><span>
</span></td></tr><tr><td>12</td><td><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>;
</span></td></tr><tr><td>13</td><td><span>}
</span></td></tr></tbody></table></code></pre>
<p>and try running the compiled output of said program:</p>
<pre data-lang="console" style="background-color:#2b303b;color:#c0c5ce;" class="language-console "><code class="language-console" data-lang="console"><span>$ </span><span style="color:#96b5b4;">.</span><span>/membug
</span><span>enter your name: jack
</span><span>hello jack
</span><span>my_perfect_string: what a beautiful string
</span></code></pre>
<p>works on my machine boss! code review +1, and merged...</p>
<p>...until our good friend <a href="https://en.wikipedia.org/wiki/Hubert_Blaine_Wolfeschlegelsteinhausenbergerdorff_Sr.">Hubert Blaine Wolfeschlegelsteinhausenbergerdorff Sr.</a> comes along. he emails me a strange error he's running into:</p>
<pre data-lang="console" style="background-color:#2b303b;color:#c0c5ce;" class="language-console "><code class="language-console" data-lang="console"><span>$ </span><span style="color:#96b5b4;">.</span><span>/membug
</span><span>enter your name: Hubert Blaine Wolfeschlegelsteinhausenbergerdorff Sr.
</span><span>hello Hubert Blaine Wolfeschlegelsteinhausenbergerdorff Sr.
</span><span>my_perfect_string: hausenbergerdorff Sr.
</span></code></pre>
<p><em><strong>note:</strong></em> if you compile and run this on your machine, you may not get the same output. that's because we're invoking <em>undefined behaviour</em> here, so the compiler can kind of do whatever it wants. I'll always provide the output that demonstrates what I'm trying to show when giving examples like this. for what it's worth, I'm running <code>clang 10.0.0-4ubuntu1</code> with target <code>x86_64-pc-linux-gnu</code>. compilation options, the <code>Makefile</code>, and such are available <a href="https://github.com/jackbondpreston/jackbondpreston.github.io/tree/master/_posts/cheri/code">code subdirectory of this article's source</a>.</p>
<p>that's not supposed to happen! his name has spilled over into our <code>my_perfect_string[]</code> array! turns out our issue is that when we use <code>fgets(char *str, int count, FILE *stream)</code>, we've set the second parameter (<code>size</code>) to <code>1000</code> - but our <code>user_name[32]</code> array can only fit 32 characters (and the last of these should be a null terminator, so 31 usable characters).</p>
<p><code>fgets()</code> fills up <code>user_name</code>, but it hasn't finished with the name yet! it doesn't care (or know) that <code>user_name</code> is full, it's just going to keep going until it finishes our user input, or reads 999 characters from standard input. thus it keeps mindlessly writing, overwriting the section memory we've used to store our precious perfect string (which happens to be immediately after <code>user_name</code>).</p>
<p><em><strong>note:</strong></em> <code>fgets()</code> has a cousin, <code>gets(char *s)</code>, which is particularly poor with regards to memory safety (due to lack of <code>size</code> parameter), <a href="https://linux.die.net/man/3/fgets">and has largely been moved away from in modern C</a>:</p>
<blockquote>
<p>LSB deprecates <code>gets()</code>. POSIX.1-2008 marks <code>gets()</code> obsolescent. ISO C11 removes the specification of <code>gets()</code> from the C language, and since version 2.16, glibc header files don't expose the function declaration if the <code>_ISOC11_SOURCE</code> feature test macro is defined.</p>
</blockquote>
<p>let's take a look at the stack in GDB to see how this happens:</p>
<pre data-lang="gdb" style="background-color:#2b303b;color:#c0c5ce;" class="language-gdb "><code class="language-gdb" data-lang="gdb"><span>(gdb) b memdebug.c:7
</span><span>(gdb) run
</span><span>Breakpoint 1, main () at membug.c:7
</span><span>7 printf("enter your name: ");
</span><span>(gdb) n
</span><span>8 fgets(user_name, 1000, stdin); // get user's name from stdin
</span><span>(gdb) n
</span><span>9 printf("hello %s", user_name);
</span><span>(gdb) x/56bc $sp
</span><span>0x7fffffffdbf0: 106 'j' 97 'a' 99 'c' 107 'k' 10 '\n' 0 '\000' 0 '\000' 0 '\000'
</span><span>0x7fffffffdbf8: 77 'M' 82 'R' 85 'U' 85 'U' 85 'U' 85 'U' 0 '\000' 0 '\000'
</span><span>0x7fffffffdc00: -24 '\350' -78 '\262' -5 '\373' -9 '\367' -1 '\377' 127 '\177' 0 '\000' 0 '\000'
</span><span>0x7fffffffdc08: 0 '\000' 82 'R' 85 'U' 85 'U' 85 'U' 85 'U' 0 '\000' 0 '\000'
</span><span>0x7fffffffdc10: 119 'w' 104 'h' 97 'a' 116 't' 32 ' ' 97 'a' 32 ' ' 98 'b'
</span><span>0x7fffffffdc18: 101 'e' 97 'a' 117 'u' 116 't' 105 'i' 102 'f' 117 'u' 108 'l'
</span><span>0x7fffffffdc20: 32 ' ' 115 's' 116 't' 114 'r' 105 'i' 110 'n' 103 'g' 0 '\000'
</span></code></pre>
<p>we can see our two character arrays are right next to each other on the stack (<code>user_name</code> contains some gibberish as it is not zero-initialised).</p>
<p><em><strong>note:</strong></em> this code was compiled with <code>-fno-stack-protector</code> to reproduce this behaviour. compilers have certain techniques which can help protect against such attacks, but there are often ways around these by using less primitive attacks. we are just ignoring these in this article for simplicity.</p>
<p>okay, at least it's a pretty easy fix: we just need to change the <code>fgets()</code> parameter <code>size</code> to <code>32</code>.</p>
<p><em><strong>note:</strong></em> you may initially think "why not <code>31</code>? don't we need to save a character for the null byte at the end?". thankfully, <code>fgets</code> does this for us. excerpt from <code>man fgets</code>:</p>
<blockquote>
<p><code>"fgets() reads in _at most one less than size_ characters from stream and stores them into the buffer pointed to by s [...] A terminating null byte ('\0') is stored after the last character in the buffer".</code></p>
</blockquote>
<p>this is a good question to be asking though, being careful is key when it comes to these kinds of things.</p>
<h3 id="why-hardware">why hardware?</h3>
<p>okay, that wasn't too bad. why are we talking about doing anything in hardware here? just write the code correctly!</p>
<p>we've looked at a very simplistic situation, with no real stakes and nothing to exploit (and an unrealistically simple bug). if this bug was exploitable for malicious gain, it could already be too late by the time we found it.</p>
<p>memory safety problems make up the vast majority of problematic security issues. the Chromium project <a href="https://www.chromium.org/Home/chromium-security/memory-safety/">found 70% of its serious security bugs were memory safety related</a> and <a href="https://msrc-blog.microsoft.com/2019/07/16/a-proactive-approach-to-more-secure-code/">Microsoft found the same prevalence</a>. some memory safety bugs can be incredibly complicated and go unnoticed for decades. the C language especially gives the programmer many, many opportunities to make mistakes - and it only takes one to be a problem. a lot of the software we are using these days is based on layers upon layers of software written in different languages, and there are going to be bugs in there. CHERI aims to give us some protection at a hardware level.</p>
<p><em><strong>Note:</strong></em> some languages (e.g. Rust) are going to offer you strong memory safety guarantees at compile-time, but I'm not going to include the discussions around this and how it compares to CHERI in this article. this article will focussed on how CHERI applies to C (and to some extent, C++ by extension).</p>
<h2 id="pointers-recap">pointers recap<a href="#pointers-recap" aria-label="Anchor link for: pointers-recap" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p>let's quickly recap a basic idea of what a pointer is. we're going to ignore things like <a href="https://en.wikipedia.org/wiki/Virtual_memory">virtual memory</a> for brevity. we can think of a pointer in a normal 64-bit architecture (e.g. AArch64) simply as a 64-bit unsigned value that holds the memory address of something we care about. this is a simplification (as are most things), but it can help us reason about the general idea:</p>
<pre data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><span style="color:#b48ead;">int</span><span> val = </span><span style="color:#d08770;">1593</span><span>;
</span><span style="color:#b48ead;">int </span><span>*x = &val; </span><span style="color:#65737e;">// x points to val
</span></code></pre>
<p><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1920 314"><defs><style>.prefix__prefix__d{fill:none;stroke-miterlimit:10}.prefix__prefix__f,.prefix__prefix__h,.prefix__prefix__i{font-size:24px}.prefix__prefix__f,.prefix__prefix__h,.prefix__prefix__k{fill:#fcfcfc}.prefix__prefix__f,.prefix__prefix__l{font-family:Source Code Pro}.prefix__prefix__d{stroke:gray;stroke-width:4px}.prefix__prefix__h,.prefix__prefix__m{font-family:Source Code Pro;font-weight:700}.prefix__prefix__i{fill:gray}</style></defs><g id="prefix__prefix__a"><path fill="#0c1114" d="M0 0h1920v314H0z"/><text class="prefix__prefix__h" transform="translate(577.46 133.41)"><tspan x="0" y="0">int *x</tspan></text><text class="prefix__prefix__f" transform="translate(490.97 177.1)"><tspan x="0" y="0">0x0000010000000004</tspan></text><path d="M481.16 206v18.5M760.5 206v18.5m-279 0h279" stroke="#fcfcfc" fill="none" stroke-miterlimit="10" stroke-linecap="square" stroke-width="3"/><text transform="translate(578.78 241.33)" font-size="20" font-family="Source Code Pro" fill="#fcfcfc"><tspan x="0" y="0">address</tspan></text><path stroke-width="4" stroke="#fcfcfc" fill="none" stroke-miterlimit="10" d="M752 171h204.56"/><path class="prefix__prefix__k" d="M948.64 182.62L992 171.01l-43.36-11.63v23.24z"/><text transform="translate(1272.76 177.16)" fill="#fcfcfc" font-size="24"><tspan class="prefix__prefix__m" x="0" y="0">mem[</tspan><tspan class="prefix__prefix__l" x="57.6" y="0">0x0000010000000004</tspan><tspan class="prefix__prefix__m" x="316.79" y="0">]</tspan></text><text class="prefix__prefix__i" transform="translate(1272.76 133.16)"><tspan class="prefix__prefix__m" x="0" y="0">mem[</tspan><tspan class="prefix__prefix__l" x="57.6" y="0">0x0000010000000000</tspan><tspan class="prefix__prefix__m" x="316.79" y="0">]</tspan></text><text class="prefix__prefix__i" transform="translate(1271.76 224.16)"><tspan class="prefix__prefix__m" x="0" y="0">mem[</tspan><tspan class="prefix__prefix__l" x="57.6" y="0">0x0000010000000008</tspan><tspan class="prefix__prefix__m" x="316.79" y="0">]</tspan></text></g><g id="prefix__prefix__b"><path class="prefix__prefix__d" d="M1260 58v48H985V58"/><path d="M1258 195v40H987v-40h271m4-4H983v48h279v-48zm-4-84v40H987v-40h271m4-4H983v48h279v-48z" fill="gray"/><path class="prefix__prefix__k" d="M756.16 150.93v40h-271v-40h271m4-4h-279v48h279v-48zM1258 151v40H987v-40h271m4-4H983v48h279v-48z"/><text class="prefix__prefix__f" transform="translate(1094 177.09)"><tspan x="0" y="0">1593</tspan></text><text class="prefix__prefix__h" transform="translate(1007.6 45.16)"><tspan x="0" y="0">memory (as ints)</tspan></text><path class="prefix__prefix__d" d="M1260 284v-48H985v48"/></g></svg></p>
<p>and on these normal architectures, this pointer generally is just a number. we can do weird things with it, treating it as a number...</p>
<pre data-linenos data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><table><tbody><tr><td>1</td><td><span style="color:#b48ead;">#include </span><span><</span><span style="color:#a3be8c;">stdio.h</span><span>>
</span></td></tr><tr><td>2</td><td><span>
</span></td></tr><tr><td>3</td><td><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>() {
</span></td></tr><tr><td>4</td><td><span> </span><span style="color:#b48ead;">int</span><span> magic = </span><span style="color:#d08770;">9999</span><span>;
</span></td></tr><tr><td>5</td><td><span> (</span><span style="color:#b48ead;">void</span><span>)magic; </span><span style="color:#65737e;">// suppress unused warning
</span></td></tr><tr><td>6</td><td><span>
</span></td></tr><tr><td>7</td><td><span> </span><span style="color:#b48ead;">int</span><span> arr[] = { </span><span style="color:#d08770;">1234</span><span>, </span><span style="color:#d08770;">5678 </span><span>};
</span></td></tr><tr><td>8</td><td><span>
</span></td></tr><tr><td>9</td><td><span> </span><span style="color:#b48ead;">int </span><span>*x = &(arr[</span><span style="color:#d08770;">0</span><span>]); </span><span style="color:#65737e;">// x is a pointer to first element of arr
</span></td></tr><tr><td>10</td><td><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">*(</span><span style="color:#d08770;">%lx</span><span style="color:#a3be8c;">)=</span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\n</span><span>", (size_t) x, *x);
</span></td></tr><tr><td>11</td><td><span>
</span></td></tr><tr><td>12</td><td><span> </span><span style="color:#b48ead;">unsigned long</span><span> x_addr = (size_t) x; </span><span style="color:#65737e;">// we're going to assume size_t = unsigned long here
</span></td></tr><tr><td>13</td><td><span> x_addr += </span><span style="color:#d08770;">4</span><span>; </span><span style="color:#65737e;">// sizeof(int) == 4
</span></td></tr><tr><td>14</td><td><span> x = (</span><span style="color:#b48ead;">int </span><span>*) x_addr;
</span></td></tr><tr><td>15</td><td><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">*(</span><span style="color:#d08770;">%lx</span><span style="color:#a3be8c;">)=</span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\n</span><span>", (size_t) x, *x);
</span></td></tr><tr><td>16</td><td><span>
</span></td></tr><tr><td>17</td><td><span> x_addr += </span><span style="color:#d08770;">4</span><span>;
</span></td></tr><tr><td>18</td><td><span> x = (</span><span style="color:#b48ead;">int </span><span>*) x_addr;
</span></td></tr><tr><td>19</td><td><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">*(</span><span style="color:#d08770;">%lx</span><span style="color:#a3be8c;">)=</span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\n</span><span>", (size_t) x, *x);
</span></td></tr><tr><td>20</td><td><span>
</span></td></tr><tr><td>21</td><td><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>;
</span></td></tr><tr><td>22</td><td><span>}
</span></td></tr></tbody></table></code></pre>
<p>...and this code will often still work:</p>
<pre data-lang="console" style="background-color:#2b303b;color:#c0c5ce;" class="language-console "><code class="language-console" data-lang="console"><span>$ </span><span style="color:#96b5b4;">.</span><span>/ptrs_as_numbers
</span><span>*(7fff98640c20)=1234
</span><span>*(7fff98640c24)=5678
</span><span>*(7fff98640c28)=9999
</span></code></pre>
<p>yikes! now, when you start messing with pointers like this, you're bound to run into a bunch of undefined behaviour. but C programmers write undefined behaviour all the time (and not always by accident), and my computer executes this program fine without complaining at all. doesn't it feel a bit weird that we can take a pointer to <code>arr[0]</code> and modify it to load <code>magic</code>? they're not even part of the same array...</p>
<h2 id="introducting-capabilities">introducting capabilities<a href="#introducting-capabilities" aria-label="Anchor link for: introducting-capabilities" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p>CHERI introduces capabilities, which can be thought of as an extension to pointers. they still store an address of something we care about, but they have extra information too! in a 64-bit system, a pointer would typically be a 64-bit value (as dicussed previously). the corresponding capability in a CHERI platform is 128 bits (or 129 bits if you look at it a certain way, more about that later).</p>
<p>as you might have guessed, this "extra information" takes up extra bits of the capability (how much is actually a little complicated - we will touch on that). bits are assigned to three key pieces of metadata: <em>bounds</em>, <em>permissions</em>, and <em>object type</em>. there is also an additional 1-bit <em>tag</em> which is stored out-of-band: it is not a 129-bit value - instead each 128-bit capability can be thought of as being associated with a 1-bit validity tag. the architecture manages this association for us. the diagram below is provided as a rough overview of this. note that it is not to scale.</p>
<p><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1920 314"><defs><style>.prefix__c{fill:none;stroke:#fcfcfc;stroke-linecap:square;stroke-miterlimit:10;stroke-width:3px}.prefix__f,.prefix__g{fill:#fcfcfc}.prefix__f{font-family:Source Code Pro;font-size:20px}</style></defs><g id="prefix__a"><path fill="#0c1114" d="M0 0h1920v314H0z"/><text transform="translate(101.86 232.41)" font-family="Source Code Pro" font-weight="700" fill="#fcfcfc" font-size="24"><tspan x="0" y="0">int *x (capability)</tspan></text><text transform="translate(1205.97 232.1)" font-family="Source Code Pro" fill="#fcfcfc" font-size="24"><tspan x="0" y="0">0x0000010000000004</tspan></text><path class="prefix__c" d="M1016 261v18.5M1656 261v18.5M1016 279.5h640"/><text class="prefix__f" transform="translate(1293.78 296.33)"><tspan x="0" y="0">address</tspan></text><path class="prefix__c" d="M700 191.5V173M1020 191.5V173M700 173h320"/><text class="prefix__f" transform="translate(823.78 167.74)"><tspan x="0" y="0">bounds</tspan></text><path class="prefix__c" d="M554 260.34v18.5M704 260.34v18.5M554 278.84h150"/><text class="prefix__f" transform="translate(562.78 295.68)"><tspan x="0" y="0">object type</tspan></text><g><path class="prefix__c" d="M391.89 191.56v-18.5M541.89 191.56v-18.5M391.89 173.06h150"/></g><text class="prefix__f" transform="translate(400.67 167.8)"><tspan x="0" y="0">permissions</tspan></text><text class="prefix__f" transform="translate(304.67 31.07)"><tspan x="0" y="0">tag (out-of-band)</tspan></text><g><path class="prefix__c" d="M391.33 55.92v-18.5M421.33 55.92v-18.5M391.33 37.42h30"/></g></g><g id="prefix__b"><path class="prefix__g" d="M1651.66 205.93v40h-632v-40h632m4-4h-640v48h640v-48z"/><path class="prefix__g" d="M1016 206v40H704v-40h312m4-4H700v48h320v-48z"/><path class="prefix__g" d="M700 206v40H558v-40h142m4-4H554v48h150v-48z"/><path class="prefix__g" d="M554 206v40h-12v-40h12m4-4h-20v48h20v-48z"/><path class="prefix__g" d="M538 206v40H396v-40h142m4-4H392v48h150v-48zM418.5 70v40h-22V70h22m4-4h-30v48h30V66z"/></g></svg></p>
<p>I am mostly going to focus on <em>bounds</em> in this article, as it is not too difficult to grasp, and the impact is fairly easy to demonstrate for some simple examples. the bounds represent an upper and lower bound on the memory region (address space) that this capability is allowed to access. if we try to use the capability to access some address outside of this range, the hardware will throw a fault - it simply won't let us do this!</p>
<p><strong><em>note:</em></strong> it is important to note that I am going to oversimplify the way the bounds are stored in this article. this especially includes the diagram above. in reality, there is a complex compression method, necessitated by the range and sizes required by bounds. this depends on the address value, alignment, etc. for now, we shouldn't need to think about this much, just know it will be managed for us. the key take-away from this is that <em>bounds can't always be 100% precise for all addresses and ranges</em>.</p>
<p>can you imagine how we can use bounds to prevent our previous memory safety bug from occurring? the key is that we can set the bounds on the capability pointing to <code>user_name</code> which we pass to <code>fgets</code>, such that the capability may only access the contents of the array. this means that when <code>fgets</code> tries to write past the end of the <code>user_name</code> array, the processor will throw a <em>capability fault</em>, and execution of our program will cease.</p>
<p>the idea behind CHERI is that we as the C programmer don't have to set up these bounds ourselves most of the time---this is something the compiler can generate code for. the compiler knows that the <code>user_name</code> array has a length of <code>32</code>, and can set the bounds accordingly on capabilities created that point to it. let's try it...</p>
<h2 id="playing-with-cheri-risc-v">playing with CHERI RISC-V<a href="#playing-with-cheri-risc-v" aria-label="Anchor link for: playing-with-cheri-risc-v" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p>unless you're lucky enough to have access to a physical Morello board, there is the issue of actually using a CHERI implementation. for this article I will be making use of the <a href="https://en.wikipedia.org/wiki/QEMU">QEMU</a> emulator to emulate a <a href="https://en.wikipedia.org/wiki/RISC-V">RISC-V</a> CHERI environment. running <a href="https://www.cheribsd.org/">CheriBSD</a> on this emulator will allow us to have a nice <a href="https://www.freebsd.org/">FreeBSD</a>-based capability-enabled environment to play around with. I'll use <a href="https://github.com/CTSRD-CHERI/cheribuild">cheribuild</a> to easily get set up (the <code>cheribuild.py</code> step will take a very long time the first time):</p>
<pre data-lang="console" style="background-color:#2b303b;color:#c0c5ce;" class="language-console "><code class="language-console" data-lang="console"><span>$ </span><span style="color:#bf616a;">sudo</span><span> apt install autoconf automake libtool pkg-config clang bison cmake \
</span><span>ninja-build samba flex texinfo time libglib2.0-dev libpixman-1-dev \
</span><span>libarchive-dev libarchive-tools libbz2-dev libattr1-dev libcap-ng-dev
</span><span>$ </span><span style="color:#bf616a;">git</span><span> clone git@github.com:CTSRD-CHERI/cheribuild
</span><span>$ </span><span style="color:#96b5b4;">cd</span><span> cheribuild
</span><span>$ </span><span style="color:#96b5b4;">.</span><span>/cheribuild.py --include-dependencies --run/ssh-forwarding-port 2222 run-riscv64-purecap
</span><span>CheriBSD/riscv (cheribsd-riscv64-purecap) (ttyu0)
</span><span>
</span><span>login: root
</span><span>root@cheribsd-riscv64-purecap:~ #
</span></code></pre>
<p>now we have our shell inside our CheriBSD emulated platform, we can start to try things out. let's compile our <code>membug</code> program again, this time with the toolchain targetting CheriBSD RISC-V - this will have been built as part of the dependencies already.</p>
<p>once our <code>membug-cheribsd</code> executable is built, we can <code>scp</code> it over to the CheriBSD filesystem. remember, we set up the SSH forwarding port to <code>1111</code>.</p>
<p>from a terminal on your host machine:</p>
<pre data-lang="console" style="background-color:#2b303b;color:#c0c5ce;" class="language-console "><code class="language-console" data-lang="console"><span>$ </span><span style="color:#bf616a;">~/cheri/output/sdk/utils/cheribsd-riscv64-purecap-clang</span><span> membug.c</span><span style="color:#bf616a;"> -Wall -g -fno-stack-protector -o</span><span> membug-cheribsd
</span><span>$ </span><span style="color:#bf616a;">scp -P</span><span> 2222 ./membug-cheribsd root@localhost:</span><span style="color:#bf616a;">~</span><span>/
</span></code></pre>
<p>and now we can see what happens when we explore our bug with CHERI:</p>
<pre data-lang="console" style="background-color:#2b303b;color:#c0c5ce;" class="language-console "><code class="language-console" data-lang="console"><span>$ </span><span style="color:#96b5b4;">.</span><span>/membug-cheribsd
</span><span>enter your name: jack
</span><span>hello jack
</span><span>my_perfect_string: what a beautiful string
</span><span>
</span><span>$ </span><span style="color:#96b5b4;">.</span><span>/membug-cheribsd
</span><span>enter your name: Hubert Blaine Wolfeschlegelsteinhausenbergerdorff Sr.
</span><span>In-address space security exception (core dumped)
</span></code></pre>
<p>it's working! we are getting a capability fault as we exceed the bounds of the
<code>user_name</code> capability bounds. we can use gdb to verify this is caused by the bounds fault:</p>
<pre data-lang="gdb" style="background-color:#2b303b;color:#c0c5ce;" class="language-gdb "><code class="language-gdb" data-lang="gdb"><span>(gdb) run
</span><span>Starting program: /root/membug-cheribsd
</span><span>enter your name: Hubert Blaine Wolfeschlegelsteinhausenbergerdorff Sr.
</span><span>
</span><span>Program received signal SIGPROT, CHERI protection violation.
</span><span>Capability bounds fault caused by register ca6.
</span><span>0x0000000040314ce8 in memcpy (dst0=0x3fffdfff44, src0=<optimized out>, length=54) at /home/jack/cheri/cheribsd/lib/libc/string/bcopy.c:143
</span><span>(gdb) p $ca6
</span><span>$1 = () 0x3fffdfff78 [rwRW,0x3fffdfff44-0x3fffdfff64]
</span></code></pre>
<p>as we can see, the bounds for our <code>user_name</code> capability (which is stored in capability register <code>ca6</code>) are <code>0x3fffdfff44-0x3fffdfff64</code>, but the address is <code>0x3fffdfff78</code>. this is out of the bounds allowed by the capability, so the architecture throws a fault. if we look at the assembly generated by the compiler, we can see it set our capability bounds to a size of 32 to enforce this behaviour:</p>
<pre data-lang="asm" style="background-color:#2b303b;color:#c0c5ce;" class="language-asm "><code class="language-asm" data-lang="asm"><span style="color:#8fa1b3;">0000000000001ce8 <main>:
</span><span style="color:#65737e;">; int main() {
</span><span style="color:#8fa1b3;"> cincoffset csp</span><span>, </span><span style="color:#8fa1b3;">csp</span><span>, -</span><span style="color:#d08770;">160
</span><span style="color:#8fa1b3;"> csc cra</span><span>, </span><span style="color:#d08770;">144 </span><span style="color:#8fa1b3;">(csp)
</span><span style="color:#8fa1b3;"> csc cs0</span><span>, </span><span style="color:#d08770;">128 </span><span style="color:#8fa1b3;">(csp)
</span><span style="color:#8fa1b3;"> cincoffset cs0</span><span>, </span><span style="color:#8fa1b3;">csp</span><span>, </span><span style="color:#d08770;">160
</span><span style="color:#8fa1b3;"> cincoffset ca0</span><span>, </span><span style="color:#8fa1b3;">cs0</span><span>, -</span><span style="color:#d08770;">36
</span><span style="color:#8fa1b3;"> csetbounds ca2</span><span>, </span><span style="color:#8fa1b3;">ca0</span><span>, </span><span style="color:#d08770;">4
</span><span style="color:#8fa1b3;"> cincoffset ca0</span><span>, </span><span style="color:#8fa1b3;">cs0</span><span>, -</span><span style="color:#d08770;">60
</span><span style="color:#8fa1b3;"> csetbounds ca0</span><span>, </span><span style="color:#8fa1b3;">ca0</span><span>, </span><span style="color:#d08770;">24
</span><span style="color:#8fa1b3;"> csc ca0</span><span>, -</span><span style="color:#d08770;">128 </span><span style="color:#8fa1b3;">(cs0)
</span><span style="color:#8fa1b3;"> cincoffset ca1</span><span>, </span><span style="color:#8fa1b3;">cs0</span><span>, -</span><span style="color:#d08770;">92
</span><span style="color:#8fa1b3;"> csetbounds ca1</span><span>, </span><span style="color:#8fa1b3;">ca1</span><span>, </span><span style="color:#d08770;">32
</span><span style="color:#8fa1b3;"> csc ca1</span><span>, -</span><span style="color:#d08770;">144 </span><span style="color:#8fa1b3;">(cs0)
</span><span style="color:#8fa1b3;"> mv a1</span><span>, </span><span style="color:#8fa1b3;">zero
</span><span style="color:#8fa1b3;"> csd a1</span><span>, -</span><span style="color:#d08770;">104 </span><span style="color:#8fa1b3;">(cs0)
</span><span style="color:#8fa1b3;"> csw a1</span><span>, </span><span style="color:#d08770;">0 </span><span style="color:#8fa1b3;">(ca2)
</span></code></pre>
<h3 id="chains-of-capabilities">chains of capabilities</h3>
<p>at this point you may be thinking "okay, that's great, but if we can just set the bounds of a capability with an instruction then what's the point? surely I can just set global bounds on some random pointer and access whatever I want?"</p>
<p>fundamental to the idea of capabilities is their <em>provenance</em> and <em>monotonicity</em>.</p>
<p><em>provenance</em>, simply put, means we can only construct a capability from an existing capability, using specific instructions. we can't just create a capability from some random <code>size_t</code> and use it to load/store something. let's see what happens when we try to run our <code>ptrs_as_numbers</code> program on CheriBSD:</p>
<pre data-lang="gdb" style="background-color:#2b303b;color:#c0c5ce;" class="language-gdb "><code class="language-gdb" data-lang="gdb"><span>(gdb) run
</span><span>Starting program: /root/ptrs_as_numbers-cheribsd
</span><span>*x=1234
</span><span>Program received signal SIGPROT, CHERI protection violation.
</span><span>Capability tag fault caused by register ca1.
</span><span>0x0000000000101c66 in main () at ptrs_as_numbers.c:1414
</span><span> printf("*x=%d\n", *x);
</span><span>(gdb) p $ca1
</span><span>$1 = () 0x3fffdfff74
</span></code></pre>
<p>we get a fault, because the tag isn't set. any capability with a tag not set to 1 cannot be dereferenced -- it is invalid. in fact, this capability has no capability metadata -- when we copied it into our <code>unsigned long</code>, we just copied the 64-bit address.</p>
<p><em>monotonicity</em> is what stops us taking an existing capability and creating a capability with more permissions and/or access than the original. it stipulates that when we create a capability from another capability (which we have to do -- provenance), the permissions and bounds of the new capability must be less than or equal to the original. so our bounds can only get narrower as we create new capabilites from an existing capability. this means that capabilities trace back in a chain - they are all created from other capabilities, and narrowed as necessary. in this case, (simplified) when the kernel loads our program it will give us capabilities that are wide enough to do everything we need to do, and the compiler will try and make sure all the capabilities that we make and use from these are as tightly bound and unpermissive as possible.</p>
<h3 id="cheri-fying-code">CHERI-fying code</h3>
<p>you'll notice we got a lot of these benefits "for free". we only had to recompile our code, and we gained this extra security. of course, CHERI does require changes to program sources. naturally, the compiler was changed a lot to implement this behaviour. in particular, CHERI also requires changes to things like the C library and kernel in order to take advantage of the features fully. sufficiently large userspace programs will generally require source changes.</p>
<p>one common issue is that a lot of existing C code assumes that <code>sizeof (*void) == sizeof(size_t)</code>. with CHERI, our pointers are now twice as big. however, <code>size_t</code> hasn't changed size, as the address space size hasn't changed - for example, if we index into an array with <code>size_t</code>, the index should still be the same size; the extra data in our <code>void *</code> capability is the metadata, not extra address data. any program that tries to convert from some <code>unsigned long</code> or <code>size_t</code> to a capability will fault - this violates provenance. so, sometimes code changes have to be made to ensure we are keeping the capability metadata around. in CHERI, we can use <code>ptraddr_t</code> to store addresses and <code>[u]intptr_t</code> to store capabilities.</p>
<p>let's make a program to see some differences in types, and demonstrate how <code>uintptr_t</code> can preserve capabilities:</p>
<pre data-linenos data-lang="c" style="background-color:#2b303b;color:#c0c5ce;" class="language-c "><code class="language-c" data-lang="c"><table><tbody><tr><td>1</td><td><span style="color:#b48ead;">#include </span><span><</span><span style="color:#a3be8c;">stdint.h</span><span>>
</span></td></tr><tr><td>2</td><td><span style="color:#b48ead;">#include </span><span><</span><span style="color:#a3be8c;">stdio.h</span><span>>
</span></td></tr><tr><td>3</td><td><span>
</span></td></tr><tr><td>4</td><td><span style="color:#b48ead;">int </span><span style="color:#8fa1b3;">main</span><span>() {
</span></td></tr><tr><td>5</td><td><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">type size (hex) size (dec)</span><span style="color:#96b5b4;">\n</span><span>");
</span></td></tr><tr><td>6</td><td><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">=====================================</span><span style="color:#96b5b4;">\n</span><span>");
</span></td></tr><tr><td>7</td><td><span style="color:#b48ead;">#ifdef</span><span> __PTRADDR_TYPE__
</span></td></tr><tr><td>8</td><td><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">ptraddr_t 0x</span><span style="color:#d08770;">%.2lx %.2lu</span><span style="color:#96b5b4;">\n</span><span>", sizeof(ptraddr_t), sizeof(ptraddr_t));
</span></td></tr><tr><td>9</td><td><span style="color:#b48ead;">#endif
</span></td></tr><tr><td>10</td><td><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">uintptr_t 0x</span><span style="color:#d08770;">%.2lx %.2lu</span><span style="color:#96b5b4;">\n</span><span>", sizeof(uintptr_t), sizeof(uintptr_t));
</span></td></tr><tr><td>11</td><td><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">size_t 0x</span><span style="color:#d08770;">%.2lx %.2lu</span><span style="color:#96b5b4;">\n</span><span>", sizeof(size_t), sizeof(size_t));
</span></td></tr><tr><td>12</td><td><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">void* 0x</span><span style="color:#d08770;">%.2lx %.2lu</span><span style="color:#96b5b4;">\n</span><span>", sizeof(</span><span style="color:#b48ead;">void</span><span>*), sizeof(</span><span style="color:#b48ead;">void</span><span>*));
</span></td></tr><tr><td>13</td><td><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">=====================================</span><span style="color:#96b5b4;">\n</span><span>");
</span></td></tr><tr><td>14</td><td><span>
</span></td></tr><tr><td>15</td><td><span style="color:#b48ead;">#ifdef</span><span> __CHERI__
</span></td></tr><tr><td>16</td><td><span> </span><span style="color:#b48ead;">int</span><span> x = </span><span style="color:#d08770;">111</span><span>;
</span></td></tr><tr><td>17</td><td><span> </span><span style="color:#b48ead;">int</span><span> y[] = { </span><span style="color:#d08770;">888</span><span>, </span><span style="color:#d08770;">999 </span><span>};
</span></td></tr><tr><td>18</td><td><span>
</span></td></tr><tr><td>19</td><td><span> </span><span style="color:#b48ead;">int </span><span>*a = &x;
</span></td></tr><tr><td>20</td><td><span> </span><span style="color:#b48ead;">int </span><span>*b = &(y[</span><span style="color:#d08770;">0</span><span>]);
</span></td></tr><tr><td>21</td><td><span>
</span></td></tr><tr><td>22</td><td><span> </span><span style="color:#65737e;">// transplant address from capability a to capability b
</span></td></tr><tr><td>23</td><td><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">*b: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\n</span><span>", *b);
</span></td></tr><tr><td>24</td><td><span> ptraddr_t a_addr = </span><span style="color:#bf616a;">__builtin_cheri_address_get</span><span>(a);
</span></td></tr><tr><td>25</td><td><span> b = </span><span style="color:#bf616a;">__builtin_cheri_address_set</span><span>(a, a_addr);
</span></td></tr><tr><td>26</td><td><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">*b: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\n</span><span>", *b);
</span></td></tr><tr><td>27</td><td><span>
</span></td></tr><tr><td>28</td><td><span> b = &(y[</span><span style="color:#d08770;">0</span><span>]);
</span></td></tr><tr><td>29</td><td><span> </span><span style="color:#65737e;">// uintptr_t is an unsigned integer type that preserves capabilities
</span></td></tr><tr><td>30</td><td><span> uintptr_t b_uintptr = (uintptr_t) b;
</span></td></tr><tr><td>31</td><td><span> b_uintptr += sizeof(</span><span style="color:#b48ead;">int</span><span>);
</span></td></tr><tr><td>32</td><td><span> b = (</span><span style="color:#b48ead;">int</span><span>*) b_uintptr;
</span></td></tr><tr><td>33</td><td><span> </span><span style="color:#96b5b4;">printf</span><span>("</span><span style="color:#a3be8c;">*b: </span><span style="color:#d08770;">%d</span><span style="color:#96b5b4;">\n</span><span>", *b);
</span></td></tr><tr><td>34</td><td><span style="color:#b48ead;">#endif
</span></td></tr><tr><td>35</td><td><span>
</span></td></tr><tr><td>36</td><td><span> </span><span style="color:#b48ead;">return </span><span style="color:#d08770;">0</span><span>;
</span></td></tr><tr><td>37</td><td><span>}
</span></td></tr></tbody></table></code></pre>
<p>running this on our non-CHERI host will give us:</p>
<pre data-lang="console" style="background-color:#2b303b;color:#c0c5ce;" class="language-console "><code class="language-console" data-lang="console"><span>$ </span><span style="color:#96b5b4;">.</span><span>/ptrtypes
</span><span>type size (hex) size (dec)
</span><span>=====================================
</span><span>uintptr_t 0x08 08
</span><span>size_t 0x08 08
</span><span>void* 0x08 08
</span><span>=====================================
</span></code></pre>
<p>running this on CHERI (64-bit):</p>
<pre data-lang="console" style="background-color:#2b303b;color:#c0c5ce;" class="language-console "><code class="language-console" data-lang="console"><span>$ </span><span style="color:#96b5b4;">.</span><span>/ptrtypes-cheribsd
</span><span>type size (hex) size (dec)
</span><span>=====================================
</span><span>ptraddr_t 0x08 08
</span><span>uintptr_t 0x10 16
</span><span>size_t 0x08 08
</span><span>void* 0x10 16
</span><span>=====================================
</span><span>*b: 888
</span><span>*b: 111
</span><span>*b: 999
</span></code></pre>
<h2 id="epilogue">epilogue<a href="#epilogue" aria-label="Anchor link for: epilogue" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p>I appreciate this has been a fragmented and surface level introduction to CHERI. hopefully it has provided some education in some basic aims of CHERI regardless. potential benefits and uses for CHERI go much deeper than anything I've touched on here, so please, read more about everything - and get your hands dirty trying out messing about with qemu and CheriBSD!</p>
<p>here are some links to check out:</p>
<ul>
<li><a href="https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/">CHERI homepage @ CUCL</a></li>
<li><a href="https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-941.pdf">technical report: An Introduction to CHERI</a></li>
<li><a href="https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-947.pdf">technical report: CHERI C/C++ Programming Guide</a></li>
<li><a href="https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-951.pdf">technical report: CHERI ISAv8</a></li>
<li><a href="https://www.arm.com/architecture/cpu/morello">Morello homepage @ Arm</a></li>
<li><a href="https://developer.arm.com/documentation/ddi0606/latest">Morello Architecture Reference Manual @ Arm</a></li>
</ul>
sensor watch2022-11-14T00:00:00+00:002022-11-14T00:00:00+00:00
Unknown
https://jackbondpreston.me/sensor-watch/<h2 id="preamble">preamble<a href="#preamble" aria-label="Anchor link for: preamble" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p>some time back I was browsing <a href="https://www.crowdsupply.com/">Crowd Supply</a> when I came across <a href="https://www.crowdsupply.com/oddly-specific-objects/sensor-watch">the Sensor Watch project</a> by <a href="https://github.com/joeycastillo">Joey Castillo</a>. I had wanted some kind of "hackable" watch for a while, and had looked at things like <a href="https://www.crowdsupply.com/sqfmi/watchy">Watchy</a>, but this project hit the sweet spot for me. I love my existing F91-W, and this project was a good combination of open source with community software support. one key feature that was important to me is battery life - the Sensor Watch battery life in an average usage scenario is so long that <a href="https://twitter.com/josecastillo/status/1590066358351298560">Joey's is still going strong</a>!</p>
<p>I was excited to pick one up and start messing around with it, but the first issue I came across was availability - the delivery date for Crowd Supply orders was summer 2023 (I think they ended upbeing delivered sooner than this, not sure). on top of this, shipping and import fees made it pretty prohibitively expensive. I've always found this to be an issue with Crowd Supply as someone based in the UK, even some things designed in the UK are very expensive from Crowd Supply as they are assembled in/shipped from the US. so I decided to build one myself! of course, this is more expensive than just buying it, but this was a learning experience and <a href="https://www.reddit.com/r/AskReddit/comments/dxosj/what_word_or_phrase_did_you_totally_misunderstand/c13pbyc/">knowledge is power</a>!</p>
<h2 id="component-acquisition">component acquisition<a href="#component-acquisition" aria-label="Anchor link for: component-acquisition" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p>the first challenge was acquiring all the necessary parts to actually build one. I downloaded <a href="https://github.com/joeycastillo/Sensor-Watch/tree/main/PCB/Main%20Boards">the PCB files</a> and generated a <a href="https://en.wikipedia.org/wiki/Bill_of_materials">BOM</a> to figure out exactly what I needed to acquire. I'm sure in ordinary times this would be easy enough, but the current state of some electronics/silicon supply chains had other things to say. some parts are of course still easy to come across, e.g. 10pF 0402 caps and 10k 0603 resistors; most of the components of the Sensor Watch are this kind of commonplace part. what quickly became clear from some scouring of the internet was that my main problem was going to be two parts: the <a href="https://www.microchip.com/en-us/product/ATSAML22J18A">ATSAML22J18A-MUT</a>(the processor driving the Sensor Watch), and the <a href="https://www.hirose.com/en/product/series/FH19C__FH19SC">FH19C-9S-0.5SH(10)</a> (the connector used to attach the extra sensor boards).</p>
<h3 id="atsaml22j18a-mut">ATSAML22J18A-MUT</h3>
<p>the former of these was a fairly well discussed shortage that had been ongoing for a while. it was
<a href="https://www.crowdsupply.com/oddly-specific-objects/sensor-watch/updates/blue-boards-shipping-check-your-address-green-boards-delayed-and-other-news-of-the-watch">the driving force of the Sensor Watch Crowd Supply delay</a>. I spent quite a lot of time searching around the internet, looking at various sites on the English-speaking and Chinese-speaking web. sadly this part was clearly in very short supply, and prices could get pretty insane from vendors that did have some stock. I received quotes for unit prices that include the following (USD/GBP): $79.35, $6.56, $13.61, $6.83 (MOQ 4000), £6.45. I guess some people are desperate enough to pay $79.35 :(. I spent so long looking for them that they ended up randomly coming back in stock on <a href="https://www.microchipdirect.com/">MicrochipDirect</a>. as of the time of writing this article, <a href="https://www.microchipdirect.com/product/ATSAML22J18A-MUT">they are again out of stock</a>. the unit price I bought them for was £3.92, shipping and handling was ~£12.</p>
<h3 id="fh19c-9s-0-5sh-10">FH19C-9S-0.5SH(10)</h3>
<p>this part was out of stock everywhere I initially looked (the usual contenders for parts). I searched around in a similar manner as the ATSAML22J18A-MUT, and found some similarly wild pricing. I ended up purchasing a small quantity at a unit price of £0.44 from a website called
<a href="http://archive.today/2022.11.13-230943/https://www.dacikeys.net/">dacikeys</a>. yes, the site is actually called this. yes, the unit price is cheaper than digikey and mouser. yes, I actually received all of my order, consisting of working parts. I was definitely shocked that this happened, but sometimes bravery pays off I guess. I still can't endorse this shop.</p>
<h3 id="pcb-and-stencil">PCB and stencil</h3>
<p>for the PCB I opted to go with <a href="https://jlcpcb.com/">JLCPCB</a>. I simply uploaded the relevant gerbers, and adjusted the necessary settings. notably, <a href="https://github.com/joeycastillo/Sensor-Watch/issues/14#issuecomment-922974276%22%3E">the thickness should be 0.6mm</a> - this does narrow the choice of manufacturer (for example, OSH Park doesn't go this thin). I haven't yet ordered any sensor board PCBs, but <a href="https://www.pcbway.com/">PCBWay</a> seems to be <em>the</em> option there. The PCB turned out great, although the silkscreen is a little hard to read at this size due to lack of sharpness:</p>
<a href="https://jackbondpreston.me/images/sensor-watch/pcb.jpg">
<img src="https://jackbondpreston.me/processed_images/pcb.afc2c7e177b84872.webp" alt="a closeup of a green sensor watch PCB" />
</a><h2 id="assembly">assembly<a href="#assembly" aria-label="Anchor link for: assembly" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p>I decided to assemble myself. partially because the logistics of paying for assembly when I had to source parts from many different providers seemed like a headache, partially because I thought it would be a fun challenge and learning experience!</p>
<p>a few things were necessary to solder the components to this PCB. I'm sure someone talented could hand solder this with an iron, but I can name a lot of things I'd rather do than try to do that
(especially the <a href="https://en.wikipedia.org/wiki/Flat_no-leads_package">QFN</a> SAML) - and that list includes unpleasant things. I opted to go with
<a href="http://www.flyelectric.org.uk/hot_plate.htm">hotplate soldering</a>, which is a cheaper way to access the ease of reflow soldering. for a PCB like the Sensor Watch, where almost all the components are on one side, it's ideal. the hotplate I have is the ever-popular
<a href="https://www.miniware.com.cn/product/mhp30-mini-hot-plate-preheater/">MHP30</a>, which I run <a href="https://github.com/Ralim/IronOS">IronOS</a> on. I highly recommend it, it's great! my soldering iron is the iconic
<a href="https://pine64.com/product/pinecil-smart-mini-portable-soldering-iron/">Pinecil</a> (not the <a href="https://www.pine64.org/2022/07/28/july-update-a-pinecil-evolved/">fancy new V2</a> though :[) which also runs IronOS. nice!</p>
<h3 id="process">process</h3>
<p>the assembly process is as follows:</p>
<ul>
<li>apply solder paste to the PCB with the stencil. make sure the stencil is really flush and justkind of squeegee it on with a plastic card. I used tape to hold it in place. then carefully removed the stencil, avoiding smudging the paste in doing so.</li>
<li>place components on the PCB. this was by far the most painful part of the whole process. a steady hand is not something I was blessed with, and some of these parts are really small. I used a microscope from Amazon for this, the ample manouverable lighting was a big help. a lot of time and patience is required, but it's very first time doable with no prior experience! simply go through the parts one by one, or by area of the board - whatever you prefer. then pick up the respective part with some fine tweezers, and slowly put in place on the solder paste. thankfully, the paste will lightly stick the component in place once you've done this (it is not knock-proof though!).</li>
<li>carefully (really!) place the PCB onto the hotplate and heat up. keep on until everything seems to be melted, and the components have hopefully been pulled into place. that's the top side done! let it cool down, then move on to the bottom.</li>
<li>time for some hand soldering. the button is pretty small, and very fiddly to do. I found you don't need too much precision, but you have to be really careful with your iron as the plastic button will melt if you touch it. once that's in place, it's just a matter of
<a href="https://youtu.be/Zr0pKeC2VFU?t=185">removing the buzzer connector from your old PCB and soldering it onto the back of the Sensor Watch PCB.</a> this will feel blissfully easy after the button! you also have to place the battery clip, but no soldering needed here :).</li>
</ul>
<p>one area I found particularly difficult was the area with the oscillator crystal and the two 0402 capacitors, C7 and C8. things are a bit cramped here, so extra care was needed:</p>
<a href="https://jackbondpreston.me/images/sensor-watch/c7c8.jpg">
<img src="https://jackbondpreston.me/processed_images/c7c8.848a11200795defd.webp" alt="a closeup of a green sensor watch PCB, with an area circled. the area contains some small, and closely grouped copper pads" />
</a><h2 id="software">software<a href="#software" aria-label="Anchor link for: software" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p>at this point the watch was assembled with all components in place. did it work? at this stage, no idea. hopefully yes, and I could progress to the more familiar world of embedded software.</p>
<h3 id="bootloader">bootloader</h3>
<p>the next necessary step is to flash the bootloader, so that we can put the firmware in place. unfortunately this requires a little more real-world action. we need to access the SWD points on the board to write the bootloader. ideally you could do this with some kind of
<a href="https://en.wikipedia.org/wiki/Pogo_pin">pogo pin</a> jig - and if you were doing any number exceeding about 5 I'm sure this would be worth the time. however, I decided to just solder some jump wires (stripped on one end, solid tip female on the other) to the points on the board. they're all close, but it's easy enough to do (albeit ugly). then I connected these to my
<a href="https://www.adafruit.com/product/1501">Adafruit Trinket M0</a> (PyRuler would also work).the pin mapping is as follows: SWD=0, SWC=1, RST=3, V+=3V, GND=GND.</p>
<p>I used the
<a href="https://github.com/joeycastillo/Sensor-Watch/blob/main/utils/flash_watch_pyruler/flash_watch_pyruler.ino">flasher from the sensor watch repo to flash the bootloader</a>. note that you could build the bootloader yourself first and put the generated binary into bootloader.h - the source is located
<a href="https://github.com/joeycastillo/uf2-samdx1">here</a>. personally, I just used the prebuilt version from the repo. I had to change part of the Adafruit DAP library and add the SAM L22 DID to get this to work,
<a href="https://github.com/joeycastillo/Sensor-Watch/issues/83#issuecomment-1229353899">I provided the diff of this change</a> in a Sensor Watch GitHub issue (I just now am remembering I promised to upstream this, oops!). mercifully, I got the red blinky LED, and all was good! I unsoldered the wires from the board, and tried to clean up most of the solder blob to keep the board fairly flat.</p>
<h3 id="movement">movement</h3>
<p>now the bootloader is in place, the main firmware can be installed!
<a href="https://www.sensorwatch.net/docs/movement/">the community firmware, Movement</a>is great, so this is what I installed. there are a bunch of different useful faces available, and more functionality is always being added.
flashing firmware was easy: I plugged the PCB into the end of a USB Micro B cable (plugged on the other end into my computer) and double tapped the reset button (I find this has to be done quite quickly, using my fingernail was the trick to doing this reliably on such a small button). done successfully, the LED on the board pulses and a new drive labelled "WATCHBOOT" appears on the computer. now a built UF2 firmware file can just be dragged onto the device to flash, thanks to the bootloader flashed earlier. for the initial test, I just used a
<a href="https://www.sensorwatch.net/docs/firmware/prebuilt/">prebuilt image</a> to check everything was working. I flashed this, and the LED pulsed and turned off, signalling success.</p>
<p>from here I just assembled the watch with the Sensor Watch PCB, and it worked! I verified LED and buzzer function by playing around with various functionality. success!</p>
<h2 id="developing-on-movement">developing on movement<a href="#developing-on-movement" aria-label="Anchor link for: developing-on-movement" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p>one face I found particularly cool was the <a href="https://github.com/joeycastillo/Sensor-Watch/blob/main/movement/watch_faces/complication/totp_face.c">TOTP face</a>. I use <a href="https://en.wikipedia.org/wiki/Time-based_one-time_password">TOTP</a>
<a href="https://en.wikipedia.org/wiki/Multi-factor_authentication">2FA</a> on various accounts, so having access to the codes on my wrist at all times was really appealing. at the time, the TOTP face only supported one key - so I decided to improve it.</p>
<p>thankfully, Sensor Watch has an emulator for development. without this, development would be pretty tiresome with the flashing and reassembling of the watch getting tiring if you needed to iterate on some code and test it on the watch. the emulator runs inside the browser and uses
<a href="https://en.wikipedia.org/wiki/Emscripten">Emscripten</a>.
<a href="https://github.com/joeycastillo/Sensor-Watch#using-the-movement-framework">some minimal instructions on how to build this is available on the README</a>. this allowed me to extend the TOTP face easily and allow for multiple keys.
<a href="https://github.com/joeycastillo/Sensor-Watch/pull/84">my PR was merged</a>, and the functionality is now available for anyone to use. the keys are added at compile time, so they are baked into the firmware on flashing. for my purposes this is fine, as I never really change them. however, with the recent addition of a
<a href="https://os.mbed.com/blog/entry/littlefs-high-integrity-embedded-fs/">LittleFS</a> filesystem, the community have added <a href="https://github.com/joeycastillo/Sensor-Watch/blob/main/movement/watch_faces/complication/totp_face_lfs.c">a version of the face which stores the keys on the filesystem</a>. awesome!</p>
<p>some more details on using Sensor Watch for TOTP is available
<a href="https://blog.singleton.io/posts/2022-10-17-otp-on-wrist/">on this blog post</a>
(<a href="https://news.ycombinator.com/item?id=33243434">HN discussion, if you dare</a>). it's even running my code :)!</p>
<h2 id="epilogue">epilogue<a href="#epilogue" aria-label="Anchor link for: epilogue" class="header-link">[<img src="/images/link.svg" />]</a></h2>
<p>some summary thoughts:</p>
<ul>
<li>shoutout to Joey Castillo. for creating the Sensor Watch as a beautifully open source project (the fact I could independently make my own is what it's all about!). for being
<a href="https://github.com/joeycastillo/Sensor-Watch/issues/83">so helpful and kind</a> when I asked for help. for having such a positive attitude towards those in the community who are using Sensor Watch to learn about all kinds of things (seriously, check out the
<a href="https://discord.gg/NtMVTBNca7">Oddly Specific Objects Discord</a> to see how much this guy is giving to the community).</li>
<li>sometimes it's worth just trying things that are difficult. this is my first time successfully doing and small-scale soldering of this kind, and it worked out great with some patience. having the motivation from making something I thought was really cool was an important factor here I think.</li>
<li>if you have a Sensor Watch (or are planning to!) please go ahead and <a href="https://github.com/joeycastillo/Sensor-Watch">contribute to movement</a> if you have a cool idea. I'm sure some reviews would be helpful to spot any issues on existing PRs before a maintainer gets to them to save some time.</li>
<li>the one issue I've had with using my Sensor Watch for TOTP is clock accuracy. the clock drifts over time, so I have to set the time once or twice a week to keep it nice and accurate for the TOTP functionality to be nice to use. but a community member is working on this, and it's going to get a lot better. check out the Discord channel to see some seriously cool engineering going into this calibration effort.</li>
</ul>
<p>
<a href="https://jackbondpreston.me/images/sensor-watch/watch1.jpg">
<img src="https://jackbondpreston.me/processed_images/watch1.8eb1a1bb66f57f56.webp" alt="a shot of a yellow and black assembled sensor watch, lying horizontally on a surface" />
</a>
<a href="https://jackbondpreston.me/images/sensor-watch/watch2.jpg">
<img src="https://jackbondpreston.me/processed_images/watch2.368eeb0770ca74e2.webp" alt="a shot of a yellow and black assembled sensor watch, lying vertically on a surface" />
</a>
<a href="https://jackbondpreston.me/images/sensor-watch/wrist.jpg">
<img src="https://jackbondpreston.me/processed_images/wrist.2912484e1c6755ee.webp" alt="a shot of a yellow and black assembled sensor watch, on the author's wrist" />
</a></p>