Introduction

For the past six months, I have been switching between a MacOS computer at work, and a Debian computer at home. I do programming with both computers, and for the most part the experience is quite comparable. Both take a UNIX-like approach to their operating system after all. However, a small niggle with how MacOS approaches keyboard shortcuts has brought me some pain.

My personal computer runs Sway¹ as it’s window manager, and recently I have been using Vim² more-and-more as my text editor. Both Sway and Vim share a similar ethos when it comes to the interface they provided to users. They are designed to be controlled primarily by the keyboard via keyboard shortcuts. Both also offer a considerable amount of customisation to the user.

My Mental Model of Modifiers

With Sway in particular, it is a common convention to use the same (configurable) key as the first key in any keyboard shortcut. This key is normally known as mod in Sway’s parlance, most obviously so in the default configuration Sway provides³. A typical choice for mod is what Apple would call the “Command” key and Microsoft would call the “Windows” key. Since it is most common to do so in the Linux community, I refer to it as the “Super” key.

To me this distinction works out quite naturally. By reserving “Super” for use by the window manager exclusively, a natural hierarchy of keyboard shortcuts results.

1. “Super” is used to spawn, kill, and navigate between applications.
2. “Control” is used to navigate and control applications themselves.

This means that when my finger goes to either key, it is because I have a distinct use-case in mind. Either, I am attempting to manipulate the operating system and numerous applications I have open. Or, I am attempting to manipulate the program that is currently active.

This distinction between the “Super” and “Control” keys helps to simplify my mental model of the computer’s operation by separating concerns. For the purpose of committing these keyboard shortcuts to memory this organisation is an excellent choice because at no time am I mixing keyboard shortcuts for the operating system with those that depend on the current application.

MacOS’s Model of Modifiers

MacOS ignores the separation of concerns entirely, and as far as I can tell, there is no easy way to rectify it. The place I notice this most egregiously is when browsing with Firefox. On my home and work computers, to change tabs I use Ctrl+Tab to move to the next tab, and Ctrl+Shift+Tab to move to the previous tab. On my home computer, to close a tab I use Ctrl+w, whilst my work computer changes this to be Super+w.

My best guess for why Firefox behaves this way can be found in Apple’s Human Interface Guidelines⁴. The section on keyboards⁵ provides a list of “Standard keyboard shortcuts” which specifies both the Tab/Shift+Tab and w key behaviours Firefox implements. However, Firefox has an immediate problem. “Super”, which Apple specifies as the preferred main modifier⁵ already has systemwide bindings for Super+Tab and Super+Shift+Tab and hence falls back to “Control” as a last resort.

Conclusion

This is a disappointing limitation on the part of Apple’s human interface design. Apple prefers that designers use the “Super” key, but also uses it for their own purposes. In an ideal world and per Apple’s Guidelines, “Option” would fill the gap, but this modifier is not common outside of Apple’s ecosystem. Hence, designers such as those of Firefox have to strike a compromise between the behaviour that their users expect from other computers, and the behaviour that Apple will permit on their hardware. Apple alone does not define the “cultural conventions” of computers, and the separation of concerns between “Super” and “Control” keys seems like an obvious winner for simplifying the mental model of users.

Footnotes

What are they?

In an education context, microthemes are employed with the dual purpose of teaching particular topic and simultaneously progressing the student’s formal writing skills². Their short form reduces burden on both student and marker, and may be used as pathway to writing longer form essays².

The exact form of a microtheme varies, with suggested lengths in the range of 100-500 words², short enough to be typed on a 5 by 8 inch card³, and 5 to 10 sentences⁴ to name a few. Various structures have been suggested in the literature⁵, with different educational goals.

Why do I care?

The context I care about isn’t marking student’s papers. (Though perhaps I may one day, let’s file that in the back of the memory.) I do however, care about making sure that I retain the information that I have learnt. In my immediate – organic – memory for short term use, and in written storage (e.g. this website), for long term use.

The principles of writing a microtheme are remarkably similar to a method that I “invented” in the first year of my undergraduate degree. I called my method “toolboxing” and it consistened of writing brief articles of what I learnt in each course with the rationale that they could serve as a “toolbox” of skills for me to rely upon as a graduated engineer. I started this toolbox in a TiddlyWiki⁶ but have since moved most of my work to markdown.

I found toolboxing an effective method for synthesising what I learnt, and it proved particularly valuable when revising for tests and exams. I struggled to maintain the same level of commitment to it that I showed early in my degree due to the breadth and depth of content covered in a semester. Despite this, some form of toolboxing has always stuck in my revision habits.

So what?

It’s my intent to write some more microthemes, in the form of articles on my website. sirlilpanda would quite like it if I keep up a cadence of one a week, but I think that isn’t realistic. Especially if I wish to write up some larger projects as well. We shall see!

Appendix: A note on the sources

The primary source on microthemes should be the text³. Unfortunately, this particular text is not accessible for most people – including myself. The claims that I have made about it have been drawn from my reading of the secondary sources which reference the text²⁴⁵.

Footnotes

ST Microelectronics produces a popular line of microcontrollers known as “STM32”. STM32 are 32-bit microcontrollers based around ARM Cortex-M series processor cores. To support software developers on the STM32 platform, ST Microelectronics provides a series of software packages and libraries under the name “STM32Cube” including the integrated development environment (IDE) “STM32CubeIDE” and various hardware abstraction libraries (HALs).

Providing software packages and libraries for developers is by no means an ST Microelectronics specific feature. In fact, it seems that most microcontroller manufacturers typically offer something similar to their purchasers. Manufacturer provided libraries are a useful tool and can speed up development provided they work as intended. There are few reasons to avoid them, especially in a commercial setting where development time can (rightly or wrongly) be a significant concern.

Despite the convenience of manufacturer provided libraries, it has been on my list of projects to program a microcontroller without the convenience of a HAL or IDE for some time. This is often known as “bare metal” programming, as there is no intermediate library responsible for setting up memory or manipulating device registers. Theoretically, what I learnt about microcontrollers at university should be more than sufficient to undertake such a project. However, there is almost always a significant division between theory and practice.

This blog post covers the necessary code to bootstrap an STM32F070RB into a basic C program. The source code and development history for this project can be found in the git repository¹ up to the tag minimal-c-project. I plan to cover some of the later developments present in the repository including the use of Renode² and microcontroller peripherals in a later post. In many respects, this post is reminiscent of the notes I took whilst pursuing this project, rather than a coherent presentation of the subject for beginners. I defer to the sources I drew upon for this experiment for a better presentation of the basics.

Acknowledgement

My primary point of reference for setting up a bare metal project is the excellent series of blog posts by Vivonomicon on bare metal programming³. Their blog posts go into more depth and breadth than I cover here.

Table of Contents

• Premise
  • Acknowledgement
• Table of Contents
• Minimal C Program
  • Start-up
  • Program Layout
  • Vector Table
  • Reset Handler
• Application Code
• Extensions and Next Steps
• Footnotes

Minimal C Program

The natural first step to a bare metal project is to understand what is necessary to get the target computer to the start of the main function. The target computer in this instance is the STM32F070RB chosen simply because I happen to have a NUCLEO development board with that particular microcontroller on hand.

Start-up

We can use the target’s documentation⁴ to get an idea of how the target behaves on start-up. We’ll assume, since it’s quite typical, that the target has some sort of program counter (PC) and that it’s reset to a default value on startup. We’ll also assume that we have some way to place code into particular sections of memory on startup. How exactly that’s achieved, we’ll get to in due course.

Looking at the programming manual for the F0 series (PM0215), section 2.1 provides a “Programmer’s Model” of F0 series STM32s that use the Cortex M0 processor. Starting with the programming manual is an arbitrary choice on my part, but I found it to be more relevant as a starting point than the datasheet (DS10697) for programming related work.

Section 2.1.3 of the programming manual describes the core registers in the target. Notably, this section tells us that register 15 is the program counter and is set to the value 0x00000004 on startup. This address is also known as the “reset vector” and is a part of a larger section of memory known as the “vector table”.

The vector table (discussed in section 2.3.4 of the programmer’s manual) specifies the addresses that the processor should jump to when a certain event (e.g. an interrupt, hardware fault, or reset) occurs. Hence, the value that should be loaded into the reset vector is the address of the starting point of the application code. Per section 2.2 of the programmer’s manual the vector table is located at the very start of the microcontroller’s flash memory, in the region corresponding to program code. Where exactly the program code is physically located depends on the boot configuration of the microcontroller. Since the board I’m using is configured to use internal flash memory, this will be assumed throughout the rest of the article.

Program Layout

With a rough idea of the start-up routine established, it’s necessary to consider how instructions from the compiled program need to be laid out in memory to fit the expectations of the target computer. I am using the Arm GNU toolchain⁵ arm-none-eabi-gcc which expects a GNU linker script⁶ to describe the memory layout of the target device.

The GNU linker (and similarly so for the LLVM linker⁷, one of the most common alternatives) expects memory to be divided into regions and then sections. Memory regions describe the size and position of areas of memory within the system memory. In turn, regions are used to divide particular memory types on the target system so that data may be appropriately allocated to memory. For example, the most common two regions on a modern microcontroller would be the division between flash memory (non-volatile memory) and RAM (volatile memory). Naturally, anything that wishes to be kept between reboots of the target device should be kept in the flash memory and it is the role of the linker file to ensure that happens at link time.

Sections of memory are used to assign portions of regions of memory to specific purposes. This is what enables the linker to assign pieces of data like instructions to the flash and variables to particular places in RAM. Each section specifies a name that it may be referred to by, the data that it includes, and the region of memory that it belongs to.

For brevity’s sake, I won’t present specific code for the linker script here - it’s not terribly interesting to look at anyway. Instead the linker script that I wrote is available⁸ for those who wish to read it. The key details to note are that two memory regions are assigned for flash and RAM. To the flash, the vector table, program text (instructions), and read only data (constants) are assigned. To the RAM, the data (initialised variables), bss (uninitialised variables), and dynamic allocation region are assigned. The sections assigned to RAM take up no space in the final executable written to the flash memory on the microcontroller. Instead, they are used to arrange the memory in the startup phase of the microcontroller and to guarantee that the program does not use more memory than is available. (Or otherwise warn the programmer that they need to ease off on the memory usage.)

You may have noticed a discrepancy between the location of the vector table in the linker script and its expected location according to the reset value of the program counter. Section 2.5 of the reference manual (RM0360) clarifies why this is the case. In essence, the STM32 always maps the flash memory to addresses 0x08000000 onwards, and when configured to boot from the flash memory, these addresses are aliased into memory from 0x00000000 onwards. This moves the vector table into the position that the microcontroller expects it to be. In fact, changing boot mode corresponds to changing what region of memory is mapped into the 0x00000000-0x00007FFF memory space in general.

Vector Table

As previously mentioned, the vector table sits at the start of program memory and is responsible for pointing the microcontroller to the start of a number of specific routines. Critically, one of the important pointers that it provides is the pointer to the start of the program. For our purposes it suffices to point the remaining routines to a default handler that we can overwrite in the application code as we please.

Because this also isn’t very interesting code, it is available elsewhere⁹.

Reset Handler

On startup it is necessary to manually initialise the portions of RAM that the program is expecting per the linker file. This is handled by the reset handler pointed to by the vector table. On completion the reset handler branches to the traditional “main” function that marks the beginning of the traditional C program.

The reset handler code is available elsewhere¹⁰ for those interested.

Application Code

Only now is the microcontroller ready to run the application code. After the reset handler has run, the application code is invoked via the “main” function. In the provided code I’ve set this up (in an echo of Vivonomicon’s own choices) to count upwards indefinitely. The code can be verified via a debugger such as gdb.

Extensions and Next Steps

I’ve since extended this code to manipulate the GPIO of the STM32 F070RB. In particular, the user LED built into the devlopment board I’m using. Since I am a firm believer in simulation as a means of verification, I’ve also used Renode² to simulate the board output. For the purposes of getting this post published, I’ve decided to defer that discussion to a later date.

Some of the more recent developments I’ve worked on in the context of this project includes pre-emptive scheduling (via ARM’s Systick and PendSV interrupts). However, a good few months has already gone by since I wrote that code. More writing on that may come if I find a suitable project to incorporate it into.

Footnotes

All HTTP requests after the initial document request (i.e. the one spawned by the user navigating to the web page) are caused by the result of an earlier request requiring more information from the web server. For example, the template that all pages on this website are based on contains the a stylesheet tag in the header of the form:

<link rel="stylesheet" type="text/css" href="/css/styles.css" />

Upon reading this, the web browser requests the file /css/styles.css to correctly style the web page. This particular request is present as the second request in Figure 1. Upon receiving and interpreting /css/styles.css an additional two requests are made due to the inclusion of @import statements in the stylesheet. The associated requests are 10 and 11 in Figure 1.

@import "text.css";
@import "cards.css";

The @import statements were included for separation of concerns in the styling of the website. Decomposing code in this manner is a common strategy for managing complexity in a code base and groups related code in a logical manner for design thinking. However, this is clearly interfering with the efficient loading of the website.

There is an effective way around this issue. The framework I have built this website in, Jekyll, can insert the text of a file into another at the time of compilation. This allows source code to remain well organised whilst not sacrificing the website performance.

<style>
    /* Inserted from css/text.css */
    {% include css/text.css %}

    /* Inserted from css/styles.css */
    {% include css/styles.css %}
</style>

Loading the same page, with all stylesheet and script tags, and import statements replaced with compilation-time inclusions where-possible yields an approximately 90 ms reduction in load-time (Figure 2). The number of requests is halved from 18 to 9. This is a small but noticeable improvement in the loading of the website.

The load-time improvement is limited by the large contribution of the font-loading time. Amdahl’s law would have dictated this be the first port of call for optimisation. However, since it is my preference not to self-host the fonts there is little improvement that I can make to the font-loading time at this time.

Below is an animation I produced of the electric field around a 2.4 GHz meandering inverted-F antenna design¹. I designed the PCB in KiCAD and wrote a short script to convert an exported DXF layer into a perfect electric conductor (PEC) polygon in OpenEMS. (A method loosely inspired by this article.) This appeared to be the simplest way to produce the copper geometry I wanted, and one where I would have the option to produce a full simulation pipeline in the future if I so desire. The current state of the art for importing PCBs into openEMS seems pretty poor, so writing my own script felt like an appropriate option.

The final simulation required some tweaking to achieve. This is the first time I have designed a mesh for an FDTD by hand but is necessary for OpenEMS, fortunately some debugging quickly found where I was going wrong. Placing the part within the simulation also proved challenging due to a lack of a well defined origin in my exportation method. The results below were rendered with Paraview and animated with GIMP. OpenEMS does require additional configuration to export electric field data over-time with the use of an electric field “dump” (just in case anyone wishes to replicate this kind of experiment). I should note that the location of the antenna I have drawn on in the figure below is only approximate. (See my comment about a lack of common origin, this is the greatest limitation in my current approach to simulation.)

The future work I’d like to complete with OpenEMS would be somewhat more practical. Chief in my mind is investigating high-speed signals on PCBs. I would quite like to see the impact of different transmission line geometries impacts the characteristic impedance of a trace as well as the impacts of different stackups on signal integrity and electromagnetic emissions.

Footnotes

Sadly, this website wasn’t remotely mobile-friendly, and I liked playing with JavaScript perhaps a little bit too much. You might notice the website even tells you the current time and date (why?)

Version 2 didn’t depart from the bold colour scheme. It made some minor improvements and added a development blog. (I recall agonising over whether to call it a blog or a log, in retrospect I would have called it a log.) In fact, the only reason that I now distinguish between this version and version 1 is because that very development blog does.

Version 3 refined the design features laid down by versions 1 and 2 with a minimal landing page. I’m less fond of simple landing pages these days² but there is something that I still quite like about this look.

Version 4 simply tweaked the landing page and isn’t terribly interesting so we’ll skip straight to version 5. In Version 5 I added more information to the landing page, I particularly enjoyed the “skill bars” at the time.

Version 5 is notable since I began to use Bootstrap as my first foray into mobile-friendly design (a practice I have now since stopped in favour of my own mobile-friendly CSS.)

I wish I could skip version 6. (Though I like the background!)

Version 7 followed a very similar theme with a slightly more up-beat feel.

Version 8 was a signifigant overhaul compared to most of the previous versions. I wanted to increase the cohesive feel of the website and use it as a real show piece. This was a level of ambition above everything that came before (except perhaps the first few iterations.)

I was particularly proud of version 8, and it hung around for quite a while. However, the novelty of the flashy animations slowly but surely wore off. I stripped back the animations and attempted to move the website to a much more lightweight feel, but I never really felt entirely satisfied with the layout. After much delay I decided something had to give, thus version 9 was born.

Version 9 (the current version at the time of writing) is designed to feel simple, and ideally like little more than a piece of paper with some writing and images on it. Those of you on a desktop computer have large margins on the left and right to keep a paper-like ratio, and the navigation links are static and simply underlined at the top of the page. Only the landing page has a header image, other pages are focused only on their content. I will admit that I have still indulged in a little JavaScript to change the header image on the landing page with each visit, however that is the extent of the dynamic content (unless you happen to press a specific key on the home page…)

More changes to come no doubt…

At a minimum I decided the board needed to:

• Have pins spaced appropriately for attaching to a standard breadboard with 2.54 mm pitch.
• Be programmed and controlled over an integrated USB - preferably USB type C.
• Expose SPI, I2C, and ADC capable pins.

From these requirements, I designed a schematic in KiCAD largely based upon the design notes provided by Raspberry Pi. I selected specific parts primarily with respect to availablility on JLCPCB’s supplier LCSC, as I intended to have the boards assembled through JLCPCB. I preferred having the boards manufactured for me as the RP2040 (and my selected flash IC) have pads that would be difficult - if not impossible - to solder with the soldering equipment that I own.

For the PCB I decided that four layers would be sufficient for my purposes, especially since routing is simple on a development board where pins may be essentially arbitrarily connected to the external connectors. I used the top layer for routing signals, the bottom layer for power, and the two interal layers for 0 V. Using the two internal layers for 0 V should provide a good reference and return path for the digital signals on the top layer, and high inter-plane capacitance for the power planes poured on the bottom layer.

Small improvements could have been made with respect to the position of the termination resistors on the USB data lines and the distance of the crystal with respect to the microcontroller. In both cases reducing the distance of both would have improved signal integrity, but I traded this off against ensuring closely placed decoupling capacitors to reduce loop length and thus parasitic inductance in the supply to the microcontroller.

Once manufactured, I tested the development board by plugging it in to the USB port on my computer and was delighted to see that the power supply LEDs immediately lit up and the RP2040 presented itself as a USB mass storage device for programming. Finally, after programming the board, I was able to make a connected LED blink on and off - the “Hello World!” of the embedded systems world.

The board was a success! However, there are some issues:

• The board is awkward to use on a breadboard with only one column of breadboard pins available on one side.
• I failed to include any on-board user-controllable LEDs.
• The I/O - whilst fulfilling my original requirements - is quite limited in number and an improved layout could have increased the number of available pins.
• I failed to expose the ARM SWD programming interface.

Another revision could easily solve those issues, but I have no need for a number of new RP2040 boards at the moment. However, these issues will be good for me to keep in mind for future designs.

A cursory look through STM’s Github account shows two repositories that are immediately recognisable from CubeMX’s automated generated code. These are the CMSIS core and HAL libraries. The CMSIS core library is the same for each STM32 series and provides a standard interface for software libraries. The HAL library is series specific and provides a set of common functions for configuring and using the major peripherals such as UART, I2C, SPI and so on. The HAL library requires a configuration file to be included in the source code and a template is provided called - in my case - stm32f4xx_hal_conf_template.h and located in Inc/ of the HAL repository.

A device specific CMSIS library is also required and may be included in the same manner as the HAL and CMSIS core. In the Source/Templates/ of the device CMSIS there are startup and system initialisation code (startup_stm32fxxxxx.s and system_stm32f4xx.c in my case). These both required copying into my source directory and included in the build process.

Finally, a linker script is required for building the binaries to be uploaded to the STM32. I found a suitable linker script in the CubeF4 repository. It turns out that the CubeF4 repository is a compilation of the CMSIS and HAL libraries as well as project examples and middleware for the F4 series STM32. To reduce the inclusion of unnecessary code, I have continued to include each library seperately, but just included the CubeFX repository would also be a valid approach.

I’ve consolidated these steps below for ease of reference:

1. Include CMSIS core library.
2. Include HAL library for the STM32 series being built for and make a configuration based on the included template.
3. Include the series specific CMSIS library and copy over the startup and system initilisation code.
4. Include the linker script from the series CubeFX library.

Since we can’t easily measure phase but can magnitude — and crucially phase is needed in order to determine the original image! — methods have been developed to estimate the likely original image iteratively. Central to these algorithms is satisfaction of two opposing constriants ². Often it is easy to project a guess to satisfy either constraint but to satisfy both (in a timely manner) must be achieved iteratively.

To retrieve phase information we need two key pieces of information. These will form our constraints and we will develop projections to map a guess to match the relevant constraint. The first is the Fourier modulus, a valid estimate of the original image should have a modulus that matches the measured modulus. Second is the support of the object, in order to have constraints out number the free variables (which is a requirement for producing a unique image¹) we define areas in which the image cannot exist. For an $N$-pixel image the Fourier modulus has approximately $N/2$ free variables, so the support of the object must not exceed more than half of the image area¹.

Now from our two constraints we need form two projections. Projections are maps that apply a transformation to an estimate image that cause it to fulfill the relevant constraint with minimal change to the image. For the modulus constraint we shall call its projection $P_F(x)$ and the support constraint’s projection $P_S(x)$, where $x$ is the estimate image.

We define $P_F$ to perform the following transformation to $x$

1. Produce the fourier transform of $x$, by convention this is $X$
2. Calculate the modulus $M$ of $X$ (the absolute value of each element of $X$)
3. Divide $X$ by $M$ to normalise the values of $X$, then scale $X$ by the measured modulus
4. Inverse the fourier transform of $X$ to produce the new $x$

In Python³

def fourier_projection(image: np.ndarray, target_modulus: np.ndarray) -> np.ndarray:
    '''
        Performs a minimal modification of the passed image to match the expected Fourier modulus.
        
        Parameters
        ----------
        image: np.ndarray
            Image to perform the minimal modification on.
        target_modulus: np.ndarray
            The expected fourier modulus. These are the magnitudes that the
            pixels are scaled to match.
        Returns
        -------
        np.ndarray
            The image with minimal modification, 
            passing it to fourier_modulus should match the target modulus.
    '''
    fimage = fftn(image)
    fimage_modulus = np.abs(fimage)
    fimage = (fimage/fimage_modulus) * target_modulus
    return ifftn(fimage)

$P_S$ can be defined in far simpler terms. $P_S$ simply takes the image and zeros-out any areas that are not apart of the support of the image. In Python³ this is easily achieved by multiplying by an array where one indicates a supported area, and zero not

def support_projection(image: np.ndarray, support: np.ndarray) -> np.ndarray:
    '''
        Restricts the image to the domain of its support.
        Parameters
        ----------
        image: np.ndarray
            The image to constrict the support of.
        support: np.ndarray
            Boolean array describing areas of support.
        Returns
        -------
        np.ndarray
            The image with only supported areas.
    '''
    return image * support

With two minimal modification projections defined we can begin constructing an iterative method that produces estimates of the image that produced a certain diffraction pattern. Intuitively we can see that alternating the projections should approach a solution, however, this error reduction method often falls into false minimums where it alternates between the same two points rather than approaching a valid solution. Many alternative algorithms have been proposed that attempt avoid these false minima. One such example is ‘the difference map’ and this shall be the example we focus on.

The difference map $D(x)$ is defined

$$\begin{align*} D(x) &= x + \beta\left[P_F\circ f_S(x) - P_S\circ f_F(x)\right]\\ f_F(x) &= (1 + \gamma_F)P_F(x) - \gamma_F x\\ f_S(x) &= (1 + \gamma_S)P_S(x) - \gamma_S x \end{align*}$$

Where $\beta$ is a constant that interpolates between the two projections and $\circ$ indicates function composition. $\gamma_F$ and $\gamma_S$ are further constants typically set to $\gamma_F = 1/\beta, \gamma_S = -1/\beta$.

Implementing this in Python³

def difference_map(image: np.ndarray, modulus: np.ndarray, support: np.ndarray) -> np.ndarray:
    '''
        Executes the difference map described in [1] and [2] upon the image once.
        Parameters
        ----------
        image: np.ndarray
            The image to iterate upon.
        modulus: np.ndarray
            The fourier modulus the image should be coerced to match.
        support: np.ndarray
            The support of the image.
        Returns
        -------
        np.ndarray
            The image transformed with one iteration of the difference map.
    '''
    f_F = (1 + Y_F)*fourier_projection(image, modulus) - Y_F*image
    f_S = (1 + Y_S)*support_projection(image, support) - Y_S*image
    return image + B*(support_projection(f_F, support) - fourier_projection(f_S, modulus))

B/$\beta$, Y_F/$\gamma_F$, Y_S/$\gamma_S$ are globals and defined as in [1].

With this done it is sufficient to generate a padded guess image and iterate the difference map upon it (Figure 2).

Being able to estimate the original image with only the diffraction magnitude is super handy for imaging really small things. This is done by firing a laser of appropriate wavelength through a sample — typically a protein or crystal — and measuring the diffraction pattern.

Acknowledgements

I am indebted to Joe Chen for introducing me to and teaching me about these topics. Without his guidance on the subject I doubt I would have ever learnt anything about it. Thank you Joe for teaching me something new :)

Footnotes

Let’s consider the definition of differentiation (in one dimension)

$$f'(x) = \frac{df}{dx} = \lim\limits_{\Delta x \rightarrow 0} \frac{f(x + \Delta x) - f(x)}{\Delta x}$$

Immediately it can be seen that there is an opening for a method of differentiation where we set $\Delta x$ to be some very small value in order to approximate the derivative of $f$. While this method is effective and a relatively quick task on a computer it quickly leads to inaccurate results, especially for higher order derivatives.

So how could we do better with a computer? A numerical method feels natural given the calculation-oriented nature of computers, but maybe moving closer to how humans produce derivatives (symbolic derivation) will yield better results. In order to produce a symbolic differentiator on a computer we need to decide how to represent the function as something the computer can understand and manipulate.

This function decomposition can be achieved in Python with special objects which track the operations that are performed upon them, building a tree where the root is the final operation, and leaves are variable or constant inputs (Figure 1).

Analysing this tree as if every node is a function of its own we can see that the chain rule applies. This logical step is the basis of automatic differentiation and the step that allows computers do differentiate symbolically easily.

$$\frac{\partial f}{\partial x} = \frac{df}{da}\cdot\frac{\partial a}{\partial x} + \frac{df}{db}\cdot\frac{db}{dx}$$

It’s worth noting now that this requires the use of the definition of the partial derivative which in two dimensions for a function $f(x, y)$ like above takes the form

$$f_x(x, y) = \frac{\partial f}{\partial x} = \lim\limits_{\Delta x \rightarrow 0} \frac{f(x + \Delta x, y) - f(x, y)}{\Delta x}$$

From here we can define the general form for derivatives of common functions and operations and use the automatic differentiation process to differentiate combinations of these with the chain rule. In jmath

>>> from jmath.autodiff import x, y
>>> f = 3*x + 10*y
>>> f_x = f.d(x)
>>> f_y = f.d(y)
>>>
>>> f_x
3
>>> f_y
10

Here the Variable objects ‘x’ and ‘y’ are used to construct the Function object ‘f’. The differentiate method then applies the automatic differentiation process detailed above, constructing the partial derivatives of the sub-functions per their definition and multiplying and adding them as appropriate. Some jmath universal functions (sin, cos, ln, …)

With some tweaks and the use of the jmath Vector object it is possible to produce gradient vectors and vector valued functions like so

>>> from jmath.autodiff import x, y, z
>>> f = x*y*z
>>> grad = f.d(x, y, z)
>>> grad
Vector(y*z, x*z, z*x)

For more information about jmath see the github repository and the documentation.

Foreword

While I give a cursory introduction to some programming and linear algebra concepts I often neglect defining exact behaviours (for example how vectors behave). I assume that you the reader either already know about these or are willing to research these topics yourself.

The code referenced in this post can be found in both C++ and Rust on my Github (The latest version can be found here but there is no guarantee that it still reflects this post).

I must also thank the online book Raytracing in one Weekend, my follow along of the book taught me the working principles of raytracers and heavily guided my first raytracer written in C++ which can be found here.

In the real world light is emmited by some emmisive object, it then travels in (mostly) straight lines, bouncing off objects - where certain wavelengths are absorbed, producing different colours - until all the light is absorbed. Light that is absorbed by our eyes is turned into electrical signals that are percieved as images in the brain. Raytracing emulates this process backwards in order to make digital images. A ray is cast out from the point of view and the objects it bounces off kept track of in order to determine a colour for the direction that the ray is cast in.

In order to build a raytracer we first need to define a vector. A vector is a quantity with an associated magnitude and direction. For three dimensions it is easiest to define a vector $\vec a$ with three values $x$, $y$, $z$, one for each axis in 3D space, mathematically we denote this

$$\vec a = (x, y, z)$$

Additionaly we define a set of useful properties including itemwise addition and subtraction, scaling, vector magnitude, and the dot and cross products respectively.

$$\vec u = (a, b, c), \ \vec v = (d, e, f)$$

$$\begin{align*} \vec u + \vec v &= (a+d, b+e, c+f)\\ \vec u - \vec v &= (a-d, b-e, c-f)\\ t\vec u &= (ta, tb, tc)\\ ||\vec u|| &= \sqrt{a^2 + b^2 + c^2}\\ \vec u \cdot \vec v &= ad + be + cf\\ \vec u \times \vec v &= (bf - ce, -(af - cd), ae-bd) \end{align*} $$

In a raytracer 3D vectors are often used to define points in space and colours in addition to typical vector values.

From vectors we can then derive an equation for a line, which are commonly reffered to as rays in ray tracing. All lines have a point of origin $\vec o$, and a direction $\vec d$ they point in. By multiplying the direction vector $\vec d$ by some scalar $t$ and adding it to the origin vector $\vec o$ we can produce any point $\vec r$ on the line $R(t)$. Mathematically

$$\begin{align}\vec r = R(t) = \vec o_\text{ray} + t\vec d\end{align}$$

Now that we’ve defined rays we can cast them from our viewport and note what they intersect with in order to draw an image. For this we’ll need to define our first object that rays can intersect, since spheres are a common choice we will start with them.

Mathematically speaking a sphere is defined by all the points in space a certain distance - the radius ($r$) - from a central point ($\vec o$). Relative to the point $\vec o$ we can derive an equation that is satisfied only for points on the surface of the sphere

$$\begin{equation}x^2+y^2+z^2 = r^2\end{equation}$$

If we let $\vec p$ be some vector relative to the point $\vec o$ such that $\vec p = (x, y, z)$ we note that $\vec p\cdot \vec p = x^2+y^2+z^2$ and thus if the point $\vec p$ is on the surface of the sphere $\vec p \cdot \vec p = r^2$ by Equation 2.

Using this we can determine if a point $\vec r$ on ray as defined in Equation 1 is on the sphere. To do this we note that by the definition of $\vec p$, and Equations 1 and 2

$$\begin{align*} && \vec o_\text{circle} + \vec p &= \vec r\\ && \vec p &= \vec r - \vec o_\text{circle}\\ &&&= \vec o_\text{ray} + t\vec d - \vec o_\text{circle}\\ && r^2 &= \vec p \cdot \vec p\\ &\Rightarrow& r^2 &= (\vec o_\text{ray} + t\vec d - \vec o_\text{circle}) \cdot (\vec o_\text{ray} + t\vec d - \vec o_\text{circle})\\ \end{align*}$$

$$\begin{align}\Rightarrow t^2\vec d \cdot \vec d +2t\vec d \cdot (\vec o_\text{ray} - \vec o_\text{circle}) - (\vec o_\text{ray} - \vec o_\text{circle}) \cdot (\vec o_\text{ray} - \vec o_\text{circle}) - r^2 = 0\end{align}$$

We can solve Equation 3 with the quadratic equation in order to determine if and where a ray intersects a sphere. Using this we can create the most rudimentary form of raytracer, one that casts a ray from the camera and determines if it intersects anything, if it does then colours the pixel as specified by the object that was intersected (Figure 2).

This isn’t beautiful. In order to improve this we use a few vital techniques.

• Anti-aliasing: Take multiple samples of a pixel with random noise in each sample, this means that edges will blur slightly making images look "smoother".
• Scattering: Calculate surface normals and trace a new ray off the surface, record any interactions and modify the colour of the pixel based on furth interactions of scattered rays.

Figure 3 shows these improvements.

We can also implement different material types and define how they cause rays to scatter, these are methods I have spent only enough time on to use rather than fully understand so I defer to Raytracing in one Weekend at this time.

While spheres look very nice in our renders, a far more versatile shape is the humble triangle. It feels quite natural to define a triangle by its three vertices so this is how we will proceed. Let us name them $\vec v_1$, $\vec v_2$, and $\vec v_3$. To determine if a ray intersects a triangle let us first start by determining where a ray intersects the plane of the triangle. We know that a point $\vec r$ on a plane will satisfy the equation

$$\begin{align}\vec r \cdot \vec n = \vec r_0 \cdot \vec n\end{align}$$

Where $\vec n$ is the plane’s normal (determined by the cross product of $\vec v_2 - \vec v_1$ and $\vec v_3 - \vec v_1$), and $\vec r_0$ is a point on the plane. Since $\vec r = \vec o_\text{ray} + t\vec d$ per Equation 1

$$\begin{align*} \vec r \cdot \vec n &= \vec r_0 \cdot \vec n\\ (\vec o_\text{ray} + t\vec d) \cdot \vec n &= \vec r_0 \cdot \vec n\\ \vec o_\text{ray} \cdot \vec n + t\vec d \cdot \vec n &= \vec r_0 \cdot \vec n \end{align*}$$

$$\begin{align} \Rightarrow t = \frac{\vec r_0 \cdot \vec n - \vec o_\text{ray} \cdot \vec n}{\vec d \cdot \vec n} \end{align}$$

Plugging $t$ back into Equation 1 gives the point $\vec r$ where the ray intersects the plane. It must now be determined whether this point $\vec r$ lays within the triangle. This can be done with an angle check from each vertice.

For each vertice three vectors originating from the vertice are produced

$$\begin{align*} \vec a &= \vec v_2 - \vec v_1\\ \vec b &= \vec v_3 - \vec v_1\\ \vec c &= \vec r - \vec v_1 \end{align*}$$

Vectors $\vec a$ and $\vec b$ bound the triangle and $\vec c$ is the position of the point $\vec r$ relative to the vertice in question. For the point $\vec r$ to be in the triangle the angle - per Equation 6 - between $\vec a$ and $\vec c$ must be less than or equal to the angle between $\vec a$ and $\vec b$ for all three vertices.

$$\begin{align} \cos\theta = \frac{\vec u \cdot \vec v}{||\vec u || ||\vec v ||} \end{align}$$

In C++ this might look something like

// Triangle-Ray intersection in C++ (Paraphrased)
// Full code available: https://github.com/JHay0112/raytracing/blob/v0.1.0/triangle.h
bool triangle::hit(const ray& r) const {
    // Find the plane's normal choosing v1 as origin
    vec3 n = cross(vertices[1] - vertices[0], vertices[2] - vertices[0]);
    vec3 r_0 = vertices[0];
    // Now can produce scalar form of plane and get val of t where point is on it
    double t = dot((r_0 - r.origin()), n)/dot(r.direction(), n);
    // Check that the point is in the triangle
    // Get the point
    point3 p = r.at(t);
    // First a check from v0
    vec3 v0_v1 = vertices[1] - vertices[0];
    vec3 v0_v2 = vertices[2] - vertices[0];
    vec3 v0_p = p - vertices[0];
    double v0_angle = angle_between(v0_v1, v0_v2);
    double v0_p_angle = angle_between(v0_v1, v0_p);
    if (v0_p_angle > v0_angle)
        return false;
    // Check from v1
    ...
    // Check from v2
    ...
    // Passed this far must be in triangle
    return true;
}

With this implemented it’s possible to make images that utilise triangles like the pyramid in Figure 1.

This project immensely deepened my knowledge of C++ and computer graphics, while I’m still quite a novice this has inspired me to think bigger while programming and to reach out into some new areas. I enjoyed this experience so much for C++ that I’ve decided to do it all again in Rust. Rust is a language that I have meant to learn for sometime and so far I’ve found it to live up to the hype. While not a trivial learning curve at first I’ve found Rust’s language design rewards thoroughly written and effecient programs. The first version of my Rust raytracer (admittedly triangle free) can be found here and ongoing development here.

I’m not sure where the future of this project is headed, but I do have a list of ideas I’m going to play around with and if I build something I’m proud of no doubt will there be another blog post about it. In no particular order:

• Light emmiting objects.
• Positionable cameras.
• Threading.

As an added bonus for making it to the very end of the post, a final raytraced image (Figure 4), this one from my Rust raytracer.

'''
    Renders a sine wave animation with Python.
    Author: Jordan Hay
    Date: 26/11/2021
'''

# - Imports

import numpy as np
import cv2
import math
from jmath.approximation import differentiate

# - Main

if __name__ == "__main__":

    # X/Y Dimensious to output to
    X = 1920
    Y = 1080

    # Framerate
    frames = 60

    # Function to draw
    # Sine wave with 50 pixel amplitude rounded to integer values
    # Should be noted that it will draw "upside down" y-axis points downwards
    func = lambda x: round(50 * math.sin(x*0.05) + 540)

    # Centre of x axis
    centre = X//2
    # Function that computes how close to the centre of the x-axis something is
    centreness = lambda x: 1 - abs(x - centre)/centre

    # Starting data
    data = np.zeros((Y, X, 3), np.uint8)

    # CV2 AVI output
    output = cv2.VideoWriter('project.avi', cv2.VideoWriter_fourcc(*'DIVX'), frames, (X, Y))

    # Starting points
    x, y = (0, 0)

    while x < X:
        # Sample point
        y = func(x)
        # Insert in drawing
        data[y:y+3, round(x):round(x)+3] = (255, centreness(x) * 255, centreness(x) * 255) # YX BGR
        output.write(data)
        # Get tangent gradient
        m = differentiate(func, x).value
        # Move along inversely proportionally to gradient
        # This should result in drawing speed proportional to line length
        if m != 0:
            x += 1/m
        # Plus some extra movement here
        x += 1
        
    output.release()

v4 introduces the Unit object in jmath.units as well as some associated features. Let's see a snippet of how it behaves

# Imports
from jmath import Unit, Uncertainty
from jmath.units import define_alias

# Define Units
volt = Unit("V") # voltage
ohm = Unit("Ω") # Resistance
ampere = Unit("A") # Current

# Define an alias
# Voltage = Resistance * Current
define_alias(ohm * ampere, volt)

# Using Units
voltage_drop = Uncertainty(3, 0.2) * volt
current = Uncertainty(0.1, 0.02) * ampere
resistance = voltage_drop/current # Per Ohm's Law

>>> print(resistance)
(30 ± 8) [Ω]

Units can be defined, attach themselves to values with the '*' operator, and then resulting values are computed with operations behaving much like normal numbers. They are compatible with Python's int and float types, as well as jmath's Uncertainty.

In addition it is possible to define "aliases" for specific units with the define_alias function provided in jmath.units. An alias will override a given unit or set of units if they are produced, for SI where voltage (V) can be derived from resistance* (Ω) and current (A) as well as vice versa this proves natural.

Another useful function-define_conversion-is also provided in jmath.units. define_conversion takes two units as parameters and then either a function or a multiplier that is used to convert between the two units. Both are demonstrated in the snippet below

# MULTIPLIER METHOD
    
# Imports
from jmath import Unit
from jmath.units import define_conversion

# Define Units
kg = Unit("kg") # Mass
j = Unit("J") # Energy

# Define conversion
# Note that conversions with factors are considered invertible
define_conversion(kg, j, (3e8)**2)

mass = 3 * kg
rest_energy = mass.convert_to(j)

>>> print(rest_energy)
2.7e+17 [J]

# LAMBDA METHOD
    
# Imports
from jmath import Unit
from jmath.units import define_conversion

# Define Units
C = Unit("°C")
F = Unit("°F")

# Define conversion
# Note that with conversion functions we must define it both ways
define_conversion(C, F, lambda t: (t * 9/5) + 32)
define_conversion(F, C, lambda t: (t - 32) * 5/9)

temp_F = 35 * F
temp_C = temp_F.convert_to(C)

>>> print(temp_C)
1.6666666666666667 [°C]

Since defining heaps of units and their conversions could be quite tiresome, the SI base units and some common derived SI units are provided in jmath.units.si, and some more common units (such as the electron volt) are defined in jmath.units.other.

In addition to the brand new unit object I have also done some tidying up of other issues I found while building the unit object. Primarily these are

• Fixing the display method for the Uncertainty object, I found this to have many issues I had never seen before while getting it working with the Unit object. These issues should be resolved now.
• Fixing the documentation for the experimental graphics subpackage, it was totally missing and I'm not entirely sure why but it seems to be fixed now.
• Better type hinting in the documentation, and general documentation cleanup.

Future plans for jmath include

• New functions based on those in Python's math library that are compatible with jmath objects.
• Upgrades to the Unit class, especially for systems of units, and perhaps a better alias/conversion system.

A final reminder that the documentation for jmath can be found at https://jmath.jordanhay.com/ and that the source code is avaliable at https://github.com/JHay0112/jmath/.

*Note that voltage (V) and resistance (Ω) are themselves also derived from base units and these conversions could also be added into a jmath unit system if it was so desired.

$$\begin{align}\Delta V = IR\end{align}$$

With the ability of an Arduino to quickly and accurately sample the voltages analysis can be performed that far outstrips the ability of a human with a multimeter. To highlight this the following experiment was conducted in order to verify the equation describing the current in an Resistor and Capacitor (RC) circuit

$$\begin{align}I(t) = \frac{\varepsilon}{R}e^{-\frac{t}{RC}}\end{align}$$

Where $I(t)$ is the current as a function of time, $\varepsilon$ the voltage provided to the circuit, $R$ the resistance of the resistor in the circuit, and $C$ the capacitance of the capacitor. A general rule of thumb for RC circuits is that the capacitor will charge in 5 time units $\tau$ where 

$$\begin{align}\tau = RC\end{align}$$

Given a capacitor of size $100\times 10^{-6} F$ and a resistor $220 \Omega$, the capacitor will reach capacity in roughly $0.11 s$. An almost unobservable amount of time for a human, however these were the components selected for the analysis.

The circuit and arduino was set up per Figure 1 with analog pins A0 and A1 connected either side of the resistor, and +5V and GND pins used to power the circuit.

Figure 1 - Arduino setup with analog pins connected to either side of the resistor. Note the arduino was also used to supply 5V to the circuit.

Figure 2 - Graph with predicted and measured current of the circuit. The predicted model was made with Equation 2.

Surpsingly the data found and the model match up near perfectly, with the current dropping to near zero in roughly $100-150 ms$, as the rule of thumb predicted. The apparent lag behind the model could be due unideal component behaviour or inaccuracy in the data.

UPDATE (19/10/2021):

Code used to analyse data from Arduino is now avaliable on my Github.

*With >5V and a common ground.

At the time of publishing jmath is capable of the following:

• Simple Vector Maths (jmath.linearalgebra)
  
• Calculations with uncertainties (jmath.uncertainties)
• Some limited discrete graph application (jmath.discrete)
  

The following are still in development and are largely considered:

• Further maths involving graphs (jmath.discrete)
• Circuits based upon discrete graphs (jmath.physics.circuits)
• Mechanical simulations (jmath.physics.mechanics)
• Fractions/rational maths (jmath.fractions)
• Lines and planes based on vector maths (jmath.linearalgebra)

These are subject to change and up to date information can be found on Github.

jmath is published on Github and Python Package Index (meaning it can be installed with pip), however I only intend for it to be a personal project for the time being. Documentation can be found at https://jmath.jordanhay.com/

Update (03/11/2021): Added documentation link.

Figure 1 - Render in Fusion 360 of the new rear case.

Why? Predominantly because the position of the control board fan wasn't ideal. Its position and lack of protection left it exposed to the possibility of parts of 3D print falling in as well as cables in the control box getting tangled in the blades. While neither of these things ever occurred the likely hood of them occurring was enough to provoke me into designing a new control board box (Figure 1) (Figure 2).

Figure 2 - The new control box installed at the rear of my printer.

Figure 3 - Fan duct.

 

It is worth noting that this case is not compatible with the screen cable going by its intended path. This design choice is because my screen is broken and was hardly used in favour of Octoprint anyway. Having the fan at the front puts the fan in a safer position while still offering cooling to the parts that need it.

With this goal in mind I made my first prototype, a battery powered terrarium light based on a $3 terrarium jar and some inexpensive LED's found in my spare electronics box. This design was majorly limited by the battery, even though LED's are incredibly efficient, the CR2032 batteries I was using would be consumed in a matter of days, the LED quickly becoming dim. A switch could've helped here but remembering to turn my terrarium light on and off would quickly become a chore, and I would still be wasting batteries, albeit far slower.

My attention turned to inexpensive solar garden lights. These would solve both my issues of turning the light on and off, and supply power to it. I sourced one for $3 and started modelling a new lid for the jar around the electronics inside (Figure 2).

Figure 2 - CAD Model of body and sleeve, hole for LED and battery guard rails can be clearly seen.

The jar lid is composed of three parts, PLA body and lid, and TPU sleeve. Body and lid hold all the electronics from the garden light, and the sleeve fits around them to form a seal between the lid and the jar.

After a few hiccups in printing a final version was manufactured and the electronics adapted from the garden light. It was necessary to remove a switch and move one wire in order to get the electronics to fit in the form factor of the lid, but this went smoothly. Figure 3 shows the electronics successfully installed in the lid.

Figure 3 - Electronics installed in lid.

Figure 4 and 1 show the final product at day and night respectively.

Figure 4 - The completed terrarium solar light at day.

My criteria:

I used Fusion 360 to develop the 3D model of the case. This was my first use of Fusion 360 to design something I intended to print.

Second Iteration Installed (Without Lid)

Mark II in Fusion 360