ML Intro - A minicourse in ML for Physicist

The Imitation Game¶

The imitation game or the Turing test refers to a test for assesing machine intelligence, proposed by the British mathematician and computer scientist Alan Turing in 1950.

The test is designed to asses a machine’s ability to exhibit intelligent behaviour equivalent to, or indistinguishable from that of a human.

Figure 1:“Try this game with ChatGPT/DeepSeek”

Machine Learning: Quite crudely, machine learning can be thought of as a collection of tools and techniques that transforms data into decisions (hopefully good!!) by making some form of classification, like whether or not someone likes pineapple pizza, or some form of quantitavive predictions, like how long can someone run.

Machine Learning Classification¶

One excellent example of classification is the classification tree itself. We will learn more about them soon.

Machine Learning Regression¶

Suppose we have a pile of Shoe-size vs Height data, and we want to make some predictions.

Different Types of ML¶

Supervised ML

Linear Regression
Logistic Regression
k-nearest neighbors (k-NN)
Support Vector Machine
Decision tree
Neural Networks

Unsupervised ML

Density estimation
Clustering maps
Dimensionality reduction
Outlier detection

Re-inforcement Learning

Deep-Q learning
Self-driving car

❓ Exercise¶

Q7: How to stop the Green Squiggle from being the final answer??

Statistical Interlude: Data as Monte Carlo Samples¶

Suppose the true data-generating distribution is

p(\phi \mid \text{source}).

(1)

Here, $\phi$ denotes one data point. For example, in collider physics, $\phi$ could represent one collision event, one jet image, or one reconstructed object.

For simplicity, we often write

p(\phi \mid \text{source}) \equiv p(\phi).

(2)

If we have $N$ data points,

\{\phi_1,\phi_2,\ldots,\phi_N\},

(3)

then the i.i.d. assumption means

p(\phi_1,\phi_2,\ldots,\phi_N) = \prod_{i=1}^{N}p(\phi_i).

(4)

This is an extremely useful assumption. It says that every event is generated by the same physical/statistical rule, but each event is still a fresh random draw. In collider physics, this is a very natural way to think.

Each collision event is random, but it is not arbitrary. It is sampled from a probability distribution determined by the underlying physics, detector response, reconstruction procedure, and event selection.

Now suppose we want to compute the average value of some observable $f(\phi)$ . The exact expectation value is

\langle f(\phi)\rangle_p = \int d\phi \, p(\phi) f(\phi).

(6)

But in real life, we do not know the full distribution $p(\phi)$ . We only have a finite dataset. Therefore, we approximate the distribution by the empirical distribution

p(\phi) \approx \frac{1}{N}\sum_{i=1}^{N}\delta(\phi-\phi_i).

(7)

Substituting this into the expectation value gives

\langle f(\phi)\rangle_p = \int d\phi \, p(\phi) f(\phi) \approx \int d\phi \left[ \frac{1}{N}\sum_{i=1}^{N}\delta(\phi-\phi_i) \right] f(\phi).

(8)

Using the property of the delta function,

\int d\phi \, \delta(\phi-\phi_i)f(\phi)=f(\phi_i),

(9)

we get

\langle f(\phi)\rangle_p \approx \frac{1}{N}\sum_{i=1}^{N}f(\phi_i).

(10)

In other words,

\boxed{ \langle f(\phi)\rangle_p \approx \frac{1}{N}\sum_{i=1}^{N}f(\phi_i) }

(12)

This is the statistical heart of why more data usually helps.

A few words on Monte Carlo¶

Monte Carlo methods use randomness to estimate quantities that may be difficult to compute exactly. In this sense, a dataset is already doing something Monte Carlo-like: it gives us random samples from an unknown distribution, and we use these samples to estimate averages, probabilities, losses, and other quantities.