NumPy Math and Broadcasting

3NumPy Math: explained step by step

In NumPy Basics you did arithmetic on whole arrays. This lesson fills in the rest of the number-crunching toolbox: maths functions, summaries along an axis, broadcasting (how arrays of different shapes combine), reshaping, matrix products, and the awkward cases, missing values and number types.

Universal functions (ufuncs)

Functions such as np.sqrt, np.exp, np.log, np.abs and np.round work on every element at once and return a new array of the same shape. NumPy calls them ufuncs. Do not reach for math.sqrt on an array: the math module only understands single numbers.

import numpy as np

areas = np.array([4.0, 9.0, 16.0, 20.25])
print(np.sqrt(areas))
print(np.abs(np.array([-3, 2, -7])))
print(np.round(np.exp(np.array([0.0, 1.0, 2.0])), 2))

Summaries and the axis argument

sum, mean, median, std, var, min and max squash an array into one number, and cumsum / cumprod keep the shape and give running totals or running products. On a 2-D array, axis chooses the direction. The axis you name is the one that disappears: axis=0 collapses the rows and leaves one result per column, axis=1 collapses the columns and leaves one result per row.

import numpy as np

scores = np.array([[70, 80, 90, 60],
                   [60, 65, 70, 55],
                   [88, 95, 100, 77]])
print(scores.sum(axis=0))
print(scores.sum(axis=1))
print(np.round(scores.std(axis=1), 1))
print(np.cumsum(scores[0]))

Broadcasting: arrays of different shapes

When two shapes differ, NumPy stretches the smaller array to fit, without actually copying it. It compares the shapes from the right-hand end, and two dimensions are compatible when they are equal or when one of them is 1. A missing dimension counts as 1.

• A scalar fits anything: table * 2.
• A row of shape (3,) fits a table of shape (2, 3): the row is applied to every row of the table.
• A column of shape (2, 1) fits the same table: the column is applied across every column.

import numpy as np

table = np.array([[1, 2, 3],
                  [4, 5, 6]])
row = np.array([10, 20, 30])
col = np.array([[100], [200]])
print(table + row)
print(table + col)
print(table - table.mean(axis=0))

If no stretching can make the shapes match, you get ValueError: operands could not be broadcast together with shapes (2,3) (2,). The usual cure is to turn the 1-D array into a column with reshape(2, 1) or [:, None], or to ask a summary to keep its dimension with keepdims=True.

import numpy as np

table = np.ones((2, 3))
try:
    table + np.array([1, 2])
except ValueError as err:
    print("Error:", err)
print(table + np.array([1, 2]).reshape(2, 1))

Changing shape and making ranges

reshape(rows, cols) re-arranges the same values (use -1 for one size and NumPy works it out), .T transposes rows and columns, and ravel() flattens to 1-D. np.arange(start, stop, step) stops before stop; np.linspace(start, stop, count) includes both ends and is safer for decimal steps.

import numpy as np

months = np.arange(1, 13).reshape(4, 3)
print(months)
print(months.T.shape, months.ravel()[:5])
print(np.linspace(0, 1, 5))

Matrix products

a * b multiplies element by element. a @ b (the same as np.dot(a, b) for 2-D arrays) is the matrix product: every row of a is multiplied by the column of b and added up. The inner sizes must match: (n, k) @ (k, m) gives (n, m). A typical use is quantities @ prices, which gives one total per order.

Z-scores, clipping and percentiles

To standardise data, subtract the mean and divide by the standard deviation: (x - x.mean()) / x.std(). The result says how many standard deviations each value is from average, which puts columns with different units on the same scale. np.clip(x, low, high) limits values to a range, and np.percentile(x, 90) gives the value that 90% of the data falls below.

Missing values and number types

np.nan marks a missing float. It spreads: any sum or mean that includes a nan is nan. np.isnan() finds them, and np.nanmean, np.nansum and friends skip them. An array also has one fixed dtype, and the dtype decides the maths: / always gives floats, // keeps integers, and small integer types such as int8 silently wrap around when a result does not fit (100 + 100 becomes -56). Convert with astype() before the maths, not after.

import numpy as np

readings = np.array([21.5, np.nan, 22.5, 23.0])
print(readings.mean(), np.nanmean(readings))
print(np.isnan(readings))
small = np.array([100, 120], dtype=np.int8)
print(small + small)
print(small.astype(np.int32) + small)
print(np.array([7, 8]) / 2, np.array([7, 8]) // 2)