Seven practices, one engineering team. We design, build, benchmark and operate the compute layer beneath your hardest problems — on-premise, hybrid or sovereign cloud.
A genomics pipeline and a trading engine are not the same machine. We start from your regulatory perimeter, your data gravity and your deadlines — then design backwards.
Industries · eResearch & Academia
Fair-share across organisations that do not report to each other is the constraint we design around first. Buying the hardware is comparatively simple. Deciding whose job runs when Deakin, a CRC and a Category 1 grant all want the same GPU partition in November is the part that decides whether the facility is trusted.
The constraint we design around first
A shared facility fails politically long before it fails technically. Partners withdraw when they cannot see what they received for their contribution. The scheduler configuration is therefore a governance document that happens to be machine-readable.
We have been brought in to repair three recurring failures. A partner cannot reconcile their contribution against their usage. A large group has quietly consumed the shared pool through job-array volume. Or the scheduler was tuned for throughput, so short interactive work — the kind students and instrument pipelines depend on — never gets a slot.
We start an engagement by writing the allocation rules in plain English with the partners in the room, then implementing exactly those rules. Configuration that does not match a document somebody signed is how trust erodes.
Allocation & accounting
Slurm will implement almost any allocation model you can describe. It will not tell you which one is fair. That decision belongs to the partners, and most of the dissatisfaction we see comes from mechanisms deployed without it.
| Mechanism | What it controls | Failure mode when used alone |
|---|---|---|
| Fair-share tree | Relative priority between accounts, decayed over a configured half-life | An idle partner accrues priority and then floods the queue, stalling everyone for days |
| Hard resource quotas | An absolute ceiling on core-hours, GPU-hours or storage per project | Capacity sits idle while a project that would use it has exhausted its quota |
| Quality of service tiers | Priority, limits and preemption behaviour by class of work | Tiers proliferate until nobody, including the operators, can predict scheduling |
| Reservations | Guaranteed access for a course, a campaign or an instrument run | Reserved-but-unused nodes become the facility's largest source of waste |
| Preemptible backfill | Opportunistic use of otherwise idle capacity at low priority | Useless to any code without checkpointing, which is most code by default |
| Trackable resource weighting | Charging that reflects memory, GPU and licence consumption, not just cores | Weights set once at commissioning and never revisited as hardware generations mix |
| Allocation committee | Discretionary award of large or unusual requests against stated criteria | Becomes the only path that works, so routine access depends on a meeting |
01
Slurm's accounting database holds the evidence, so it is configured as a system of record: replicated, backed up, and retained beyond the longest grant reporting obligation. Purging accounting data to save space has ended more than one reconciliation argument badly.
02
Partner reports are generated from the accounting database by a documented query, not assembled in a spreadsheet. A partner who disputes a figure can be handed the query and the raw rows, which usually ends the dispute in minutes rather than weeks.
03
Jobs carry a project code that maps to a funded activity, so usage can be reported against the grant that paid for it. Identifiers such as ORCID for people and DOIs for output data sets let a reviewer trace compute through to publication.
04
Utilisation, queue wait by partner and reservation waste are reported on a fixed cycle whether the numbers flatter the facility or not. Selective reporting is discovered eventually, and costs more credibility than a bad quarter ever does.
Federated identity & access
Researchers arrive with an identity issued by their home institution and leave when that institution says so. A shared facility that maintains its own user database inherits every joiner, mover and leaver problem of every partner at once.
Australian research federation through the AAF, with international reach via eduGAIN, covers most of the population. The remainder — industry partners, honorary appointments, international collaborators without a federated home, and students on short placements — is where the design effort actually goes.
Federation revokes authentication promptly and tells the facility nothing about why. A researcher who moves institutions can lose access to their own in-flight work overnight. We handle that with a defined grace period, project ownership that sits with the project rather than the person, and a documented handover step at close-out.
Climate, weather & earth systems
Earth-system work is usually described as one thing. In IO terms it is at least three, and a cluster tuned for one of them will disappoint on the other two. Sizing starts by asking which of these dominates.
Atmosphere, ocean, ice and land components exchange state every model step. Performance is set by the slowest link in the halo exchange, so interconnect latency and consistent routing matter more than aggregate bandwidth. A single degraded cable can cost a fortnight of throughput without triggering an alarm.
Codes in this class — models in the ACCESS family, WRF for regional atmosphere, MOM for ocean, CICE for sea ice — have well-understood scaling characteristics. We benchmark yours at your resolution rather than quoting a published curve from a different machine.
An ensemble is many perturbed runs of the same model. Each member is modest; together they saturate the scheduler and the file system's metadata layer. The limit is rarely floating-point capability. It is usually job submission rate, directory contention and the storage tier holding the output.
Once the model has run, the work becomes reading slices of netCDF and HDF5 across time and variables, mostly from notebooks. This is random read against very large files, from users who will abandon the session if it takes two minutes to draw a map. It is the phase most often left out of the sizing.
Chunk layout is the single highest-impact decision here, and it is made when the model writes its output. A run configured for fast writes can make every later analysis slow. We look at both ends together.
Research data management
Findable, accessible, interoperable and reusable are all decided by choices made during the run — identifiers, metadata capture, file layout, retention. Retrofitting them at publication is where the cost and the frustration live.
A project's data management plan states expected volume, sensitivity, retention and destination repository. We ask for it at allocation time because those four facts determine which storage tiers the project should touch at all.
Instrument settings, model configuration, software versions and parameters are written alongside the output by the workflow. Nobody reconstructs this accurately two years later, and reviewers increasingly ask.
DOIs for data sets, ORCID for people, and persistent internal identifiers for projects and runs. Minting happens through your institutional repository or national service; our part is making the identifier travel with the data.
Scratch is fast, expensive and deliberately impermanent, with an enforced expiry the users know about. The retained record lives on a different tier with different guarantees, integrity scrubbing and a tested restore path.
Container images by digest, the workflow at a tagged revision, input checksums and the exact software environment are archived with the outputs. Reproducibility that depends on one person's home directory is not reproducibility.
Retention obligations under funder and institutional policy commonly run many years past the project. We implement the schedule, evidence each action, and include snapshots and backups, which are where undocumented copies survive.
Procurement & the funding cycle
Research infrastructure funding is granted once, spent quickly, and then operated from an ongoing budget that was estimated before anyone knew what the workload would be. This shapes every sensible design decision.
Spending the whole grant on nodes is how a facility ends up with a full machine room and no power budget in year four.
A grant that buys the maximum number of nodes at day one produces a facility that cannot be extended, cannot absorb a new accelerator generation, and has no margin for the spares and staff the next six years require. We size to leave headroom in power, cooling, rack space and network ports.
A tuned cluster with no research software engineering support delivers a fraction of its capability, because most research code is written by domain specialists under deadline. We include training, documentation written for the actual user population, and retained engineering time for code optimisation in the operating model. It is the highest-return line in the budget and the first one cut.
Sensitive research data
Most research data should be as open as possible. A minority carries obligations that open infrastructure cannot satisfy, and a shared multi-institution facility is precisely where those obligations get missed.
Australia's defence trade control framework, and the Defence and Strategic Goods List it references, can apply to research and to the supply of technology, including in some circumstances to sharing it with a foreign national inside Australia. Whether a given project is caught is a legal question for your research office, not for us.
Data about Aboriginal and Torres Strait Islander peoples, lands, languages and cultural knowledge carries governance obligations that FAIR does not address. The CARE principles — collective benefit, authority to control, responsibility and ethics — sit alongside FAIR, and in this context authority to control is the one that changes architecture.
We are infrastructure engineers, and we take direction on these matters from the communities concerned and from the researchers accountable to them. Our commitment is that the platform will not be the reason a governance decision cannot be honoured.
Questions we get asked
These come up in every shared-facility conversation. Where the answer is uncomfortable, the discomfort is the useful part.
We can make it transparent, consistent and enforced, which is what partners actually need. Fair is a judgement your governance body has to make, because it depends on whether you are rewarding contribution, scientific merit, teaching load or need. Our advice is to write that judgement down in one page of plain English, then let us implement precisely that.
For steady, tightly coupled MPI work at high utilisation, owned hardware usually wins over a five-year horizon, and grant structures favour capital anyway. For spiky, embarrassingly parallel campaigns and teaching peaks, cloud is often cheaper than the nodes you would leave idle. Most facilities we work with should do both, with the split decided by measured job shapes rather than preference. Egress charges and data gravity are the numbers that decide it.
With accounting data, not with a meeting. If the facility can produce per-account usage, queue wait and rejection reasons for the disputed period, the conversation becomes a factual one and usually resolves quickly. If it cannot, the dispute is really about the absence of evidence, and no allocation change will fix that. This is the main reason we treat the accounting database as a system of record.
It is good for a business case and bad for your users. At that level, queue waits are long and interactive or exploratory work becomes impractical, which pushes students and short-turnaround research off the facility entirely. Report utilisation together with queue wait by partner and by job size. A facility optimised purely for the utilisation number is optimised for the wrong reader.
Yes, with separate quality of service tiers, reservations around teaching timetables, and limits that prevent one runaway job array from consuming the shared pool. Teaching load is highly predictable, which makes it easy to plan for once somebody asks for the timetable. The failure case is treating a course as a normal project three days before the assessment is due.
Accounting exports from any existing system, two or three representative codes we can run, the partner agreement or heads of agreement, the retention obligations, and an honest figure for the annual operating budget. From that we return a sizing with assumptions exposed, an allocation model drafted in plain English, and a seven-year cost model including power and staff.
Let's Talk
We will profile your real job mix, draft an allocation model your partners can read, and give you a seven-year cost picture that includes power, spares and the people. If the answer is that you need less hardware than you planned, we will say so.
Australian owned and staffed. We are comfortable presenting to an allocation committee rather than a procurement panel.