A Workflow for Data Science
Jupyter notebooks are a phenomenal tool for investigating and interacting with data. The interactive nature of the notebooks allows for quick and effective interrogation of the data and more generally the prototyping of an idea. By combining interaction, the documentation, the co…
Updating Query with Soft Delete for SQLAlchemy 1.4
When working on a database, there are times where we want to delete a row, but at the same time we still want to keep around the record that the particular row was referring to. There can be many reasons for this, however the basic idea is that we want the row to generally be hid…
Handling Exceptions in Numerical Python
The numerical computing capabilities of the Python ecosystem are incredibly powerful, with the Numpy and Scipy, and Pandas packages providing high performance, well tested tools. Additionally, tools like Dask and Rapids build upon these foundational packages supporting larger dat…
Speed Through Specificity
A common criticism about the Python programming language is that it is slow, often with reference to a benchmark comparing a range of tasks. This criticism is widely addressed with articles by Jake van der Plass and Anthony Shaw being two excellent examples. While I don't disagre…
Experi: A tool for computational experiments
Abstract One of the key features of computational experiments is being able to run the experiment over a large variable space. However, in my experience there aren't tools available to assist with this, particularly in the realm of High Performance Computing (HPC), where bash arr…
Distributing a Hoomd Plugin
A piece of software I have been using in my research is Hoomd, a 'relatively' new package for running Molecular Dynamics (MD) simulations. These MD simulations have the basic premise of throwing hundreds of balls into a box and shaking it to find out what happens. The relative ne…
Scraping Data from Facebook
Competition is a strange thing making you suddenly interested in the most unusual of problems It has become a tradition of The Lancer Band, of which I am a member, to produce a video as part of our ANZAC Day commemorations. These videos have been highly successful garnering milli…
Adventures in the Python Visualisation Landscape
Calling the current python visualisation landscape fragmented would probably be an understatement, since itself requires a visualisation to even begin to comprehend. For various reasons I have been unhappy with the tool I was using for visualisation at that time and I have been s…
Creating an HDF5 Bomb
You may have heard of a zip bomb or other decompression 'bombs', which have the basic premise of containing a large volume of highly redundant data that when decompressed takes up more resources than the system can handle. Within the HDF5 file format there is support for compress…
The Perils of Packaging in Python
There are many guides to packaging a python application, including the official Python Packaging User Guide. While these guides offer step by step instructions for deploying a simple application, when deviating from a guide there can be unexpected problems. The application I have…
Quaternions for Orientation in MD
In the understanding of the dynamics of a molecule in a Molecular Dynamics (MD) Simulation the two main properties we use to understand liquid behaviour are the translational motion and the rotational motion of the atoms or molecules. These motions reflect changes in the positio…
Building interfaces with click
Running computer simulations is great. You tell the computer what to do, it sits there crunching numbers for a while. Then you can come back and look at the results. The problem with this is that most simulation packages take the simulation parameters in an input file, so running…