My Microsoft Reactor Experience: Sharing my Cloud HPC learnings

How a live demo of 100-node cluster deployment became a masterclass in making complex technology accessible

Teaching Cloud HPC to the World: My Microsoft Reactor Experience

The Global Reach of Virtual Teaching

Presenting to the Microsoft Reactor audience was a unique experience - watching attendees join from Bangladesh, Bali, Nigeria, and dozens of other locations reminded me of the truly global reach of cloud technology. Live demos always carry an element of risk (will the cluster provision correctly? will we hit quota limits?), but with careful preparation and backup recordings, these challenges become manageable.

What made this session particularly meaningful was knowing that for many viewers, this could be their first exposure to accessible HPC. Researchers at institutions without traditional supercomputing facilities, startups with limited capital budgets, and students curious about computational science were all joining to learn how cloud technology could enable their work. Azure CycleCloud represents a paradigm shift for these audiences - it’s not just another cloud service, but a gateway to computational capabilities that were previously exclusive to well-funded institutions.

Setting the Stage: Why CycleCloud Matters

I began by acknowledging that while most attendees knew about cloud computing and many had heard of HPC, the intersection remained mysterious. How do you take something as complex as a traditional supercomputer and make it work in the ephemeral world of cloud computing?

The answer lies in understanding that cloud HPC isn’t about replacing traditional supercomputing but reimagining it for a world where infrastructure is code. Azure CycleCloud serves as the bridge, allowing researchers to use familiar tools like SLURM while gaining the flexibility to scale from zero to thousands of cores in minutes.

Coming from a background working with major supercomputing facilities, I’ve seen firsthand the constraints of fixed capacity. You can’t add nodes to a physical supercomputer when demand spikes. You can’t experiment with different hardware configurations without significant capital investment. These limitations don’t exist in the cloud, and CycleCloud makes that flexibility accessible without requiring researchers to become cloud architects.

The Architecture Explanation That Clicked

One challenge in teaching cloud HPC is helping people visualize the architecture. I used analogies that would resonate with the diverse audience: the CycleCloud server as a construction foreman orchestrating deployments, the scheduler node as the project manager tracking work, and worker nodes as the construction crew that scales based on demand.

The beauty of this architecture is that it respects decades of HPC scheduler development. We’re not reinventing SLURM or PBS; we’re giving them cloud superpowers. The scheduler still thinks it’s managing a traditional cluster while CycleCloud handles the orchestration behind the scenes. Research groups can migrate existing workflows without rewriting years of accumulated scripts.

The Live Demo: Where Theory Meets Reality

The demo began with deploying a CycleCloud server from the Azure marketplace. Even this initial step provided teaching moments - explaining networking configurations and how service principals enable secure communication between CycleCloud and Azure resources.

The real engagement came when configuring the cluster template. I explained each decision: Why choose specific VM sizes for scheduler versus worker nodes? How to balance cost versus performance? What’s the difference between HTC and HPC workloads, and why does it matter for hardware selection?

When I set the maximum core count to 640 for the HTC partition, the audience’s surprise was palpable. For many, the idea of casually configuring access to hundreds of cores was revolutionary. I emphasized the importance of these limits, sharing a cautionary tale of unexpected Azure bills from forgotten maximum counts.

The Aha Moment: Watching Autoscaling in Action

The climax came when I submitted a job requiring 50 nodes and watched CycleCloud automatically provision them. The audience could see in real-time as the status changed from “acquiring” to “ready,” and the job started running.

But the real “aha moment” came afterward. As the job completed and nodes sat idle, CycleCloud began terminating them automatically after the configured timeout. The chat lit up as people realized the implications: no more paying for idle supercomputers. The system optimizes costs while maintaining performance.

This led naturally to discussing Azure Spot instances for fault-tolerant workloads, potentially saving up to 90% on compute costs. For academic researchers operating on tight budgets, this was game-changing information. I shared real numbers showing how simulations costing thousands on regular instances could run for hundreds using Spot instances.

Fielding Questions: The Real Learning Happens

The Q&A revealed the audience’s depth of interest. One participant asked about robotics integration, leading to a discussion about real-time versus batch processing. Another wondered about the learning curve for SLURM users, allowing me to emphasize that if you can use SLURM on a traditional cluster, you can use it on CycleCloud without learning anything new.

A particularly insightful question about choosing between many small nodes versus fewer large nodes for MPI applications opened discussion about Azure’s HB and HC series VMs with 200 Gbps InfiniBand networking, making them comparable to or better than many on-premises supercomputers.

The most impactful question came from someone asking about enabling this for their institution in a developing country. I explained how the pay-as-you-go model makes cloud HPC more accessible than traditional supercomputing’s massive upfront investment. Small research groups can start with just hours of compute time monthly and scale as needed.

Building Confidence Through Teaching

Teaching this technology forced me to continually refine my understanding. Every audience question pushed me to find better explanations. Isabel Negrete, who coordinated the Reactor series, encouraged me to slow down and not assume prior knowledge. This advice proved invaluable - taking time to explain fundamentals like MPI and interconnect requirements brought everyone along on the journey.

The international audience reinforced the global impact of this technology. When viewers from Bangladesh or Bali asked about specific use cases, it highlighted that we’re not just teaching technology; we’re enabling scientific discovery worldwide.

Looking Ahead: The Next Session and Beyond

I previewed the upcoming session on running GROMACS and OpenFOAM, which would showcase the true power of RDMA-enabled nodes for demanding scientific computations. I wanted the audience to understand this was just the beginning - they could integrate CI/CD pipelines, use containers for reproducibility, and implement hybrid scenarios for cloud bursting.

With Azure’s continued investment, including the recent CycleCloud Workspace for Slurm that reduces deployment from days to minutes, barriers keep falling. What once required teams of administrators can now be deployed by a single researcher in an afternoon.

The Personal Journey Continues

This Microsoft Reactor session represents one stop on my journey to democratize scientific computing. From PhD research in computational chemistry to working at major supercomputing centers to teaching cloud HPC globally, each step reinforces my belief that computational science should be accessible to anyone with curiosity and determination.

When I started my PhD, significant computational resources required major institutional affiliation or substantial grants. Today, motivated individuals can access more computational power than most universities could afford a decade ago. This isn’t just technological shift; it’s fundamental democratization.

To everyone who joined the session, asked questions, or reached out afterward - thank you. You’ve reminded me that teaching technology is about empowering people to solve problems they care about. Whether modeling protein folding, simulating climate patterns, or designing new materials, the cloud is ready for your science. Thanks to Azure CycleCloud, you don’t need to become a cloud expert to harness that power. You just need willingness to learn, experiment, and dream big about what’s possible when computational limits disappear.

The future of scientific computing is being written by researchers who refuse to be constrained by traditional boundaries. With each Reactor session, we’re adding more authors to that story. The best discoveries are yet to come, and they’ll be made by people who learned that supercomputing isn’t just for the privileged few anymore - it’s for everyone with a question worth answering.