Python for Scientists & Engineers
About This Course
Python has become the common language of scientific and engineering computing, but the useful parts of it are spread across a dozen libraries that each assume you already know the others. This course covers those libraries in the order you need them, so that five days in you can take a measurement file from whatever produced it through to a result someone else can check.
Day one is the language itself, at the pace of someone who already programs in another language: the data types, the tools, and the awkward parts that are specific to Python rather than to programming. Days two and three get data under control — reading the formats you are actually sent, analysing it in Pandas or Polars, plotting it with Plotly Express, and then dropping to NumPy arrays where a loop should have been one vectorised expression. Day four is machine learning, treated as a tool with preconditions rather than a default answer, which means as much time on validation as on fitting. Day five is the scientific stack: statistics and hypothesis testing, physical units and uncertainty, and optimisation, followed by workshops chosen to match the group.
The examples are real problems rather than toy data: fitting a curve to a long run of climate records, pricing an option by Monte Carlo simulation, counting features in aerial imagery, interpolating a value across scattered measurements where the sampling is uneven. The particular datasets vary — we choose examples that suit the group, and a private course can use yours. Where the material is opinionated it says so and gives its reasons — which regression package to reach for and which to avoid, when providing derivatives to an optimiser repays the effort, and when a statistically significant result is not a practically significant one. When a routine is too slow, you time it, work out why, and then make it faster.
Every topic mixes instruction, worked examples and hands-on exercises, and you keep the notes, datasets and worked solutions afterwards. Bringing a problem from your own work is welcome; the final afternoon in particular is easier to aim at a group whose actual problems we know.
Who This Course Is For
Research scientists, engineers and technical staff who already program and now need Python for their day-to-day computational work. It suits people moving across from Matlab, R, Fortran, C or IDL, and people who have picked up some Python informally and want to work through the ecosystem properly. If you handle instrument data, run simulations, fit models, or produce analysis that other people rely on, this course is aimed at you.
It also suits technical groups where every person has arrived at Python by a different route, so the same analysis is done five ways and none of it is easy to hand over. The five days give a mixed team one shared vocabulary and one set of tools, from reading the instrument file to publishing the result.
Prerequisites
Some familiarity with programming concepts in any language is helpful, but prior programming experience is not required. You do not need to have written Python before.
The course moves at the pace of someone who already programs in another language, such as Matlab, R, Fortran, C or IDL, and Day 1 covers the Python the rest of the week depends on. No mathematics beyond what your own field already requires is assumed, and you do not need a background in statistics or machine learning; both are introduced from the beginning.
What You'll Learn
- Load, clean and reshape data from the sources your work relies on — CSV, Excel, SQL, JSON and web APIs — using Pandas or Polars.
- Write vectorised numerical code with NumPy in place of Python loops, and apply the linear algebra routines your field depends on.
- Present results as interactive Plotly Express visualisations that other people can explore themselves.
- Fit models to measured data with SciPy, and carry physical units and measurement uncertainty through a calculation so unit errors cannot propagate silently.
- Build and validate scikit-learn models, and judge whether the results can be trusted well enough to act on.
- Diagnose slow numerical code, make it faster with the appropriate tool, and scale it beyond one core when it needs to be.
Course Syllabus
Day 1: Python basics
- The core language: variables, conditions, loops and functions
- The essential data types: strings, lists, dicts, tuples and sets
- Raising and handling exceptions
- Working effectively in Jupyter and Visual Studio Code
- Files, paths and formats that survive moving between machines
- Modules, packages and environments, including supply-chain safety
- Worked example: retrieving real-time data from a REST web API
Day 2: Handling, analysing and presenting data
- Reading and writing the formats you receive: CSV, Excel, SQL, JSON and XML
- Selecting, filtering and sorting tabular data with Pandas or Polars
- Group-by, aggregation, pivot tables and joins
- Dates, times and time-series data
- Interactive visualisation with Plotly Express
- Preview: turning an analysis into a dashboard with Streamlit
Day 3: Further data analytics
- Array and matrix computing with NumPy, and where loops disappear
- Linear algebra: eigenvalues, eigenvectors and matrix decompositions
- Monte Carlo simulation, and vectorising a simulation that runs too slowly
- Linear, nonlinear and robust regression, and choosing between the packages
- Finding the unusual points in your data: z-scores, clustering and pyOD
- Clustering with K-means and DBSCAN
Day 4: Machine learning
- What machine learning does, and where it is the wrong tool
- The scikit-learn estimator interface, and choosing an estimator
- Classification and regression on your own data
- Validation, cross-validation, and diagnosing overfitting
- Feature engineering and feature selection
- Deploying a model, and what changes when it has to run in production
Day 5: Scientific and engineering computing
Morning — the scientific stack:
- Classical statistics, distributions and hypothesis testing
- Physical units, constants and uncertainty propagation with pint
- Curve fitting and optimisation with SciPy, including constrained problems
Afternoon — specialist workshops, selected to suit the group, from:
- Image processing with scikit-image
- Digital signal processing, Fourier transforms and spectrograms
- Interpolation, and ordinary differential equations
- Dimensionality reduction with PCA, t-SNE and UMAP
- Writing faster numerical code with numexpr, Numba and Cython
- Parallel and cluster computing with Dask