Polars Primer 01: Introduction

Polars introduction

What is Polars

All code snippets use Rent_Contracts.csv as input data, sourced from lemoninabag/Rentals · Datasets at Hugging Face.

Polars is a library for interacting with and manipulating tabular data. There are other Python libraries for this purpose, such as Pandas, but Polars used two underlying technologies, Apache Arrow and Rust, that give it an edge when it comes to data processing.

1. Apache Arrow stores tabular data in memory.

2. Rust is a programming language suited for fast data processing.

Polars interacts with a data stream as two abstract types: DataFrame and LazyFrame.

1. DataFrame (Eager Execution): Polars loads data immediately from its source into memory, and every it executes every operation at once. Using DataFrames for eager execution is best for interactive exploration of data (by means of Jupyter Notebooks) or dealing with small data sets.

2. LazyFrame (Lazy Execution): Polars does not load the data stream, or perform any processing on it, until it can complete a query plan. Using this precursor step, Polars optimizes operations before generating a DataFrame to carry out its instructions. The LazyFrames used here are suited to production pipelines and large data sets.

In the following sections we discuss various ways of using the Polars API. We use Python to illustrate eager execution (DataFrame) and Rust to showcase lazy execution (LazyFrame). There is a lot more to say about DataFrames and LazyFrames but we’ll leave this off to later chapters.

Importing a CSV File

import polars as pl

rental = pl.read_csv("Rent_Contracts.csv")

When we use Python, the Polars library is imported as a whole. Rust takes a modular approach, allowing us to specify package features to include by means of Cargo.toml.

// Cargo.toml

[dependencies]
polars = { version = "0.54.4", features = ["lazy", "csv", "fmt", "strings", "temporal"] }

The following Rust code shows how we read a CSV file into a LazyFrame.

// Lazy Evaluation (LazyFrame) defers loading data, giving Polars
// the opportunity to optimize the data query beforehand.

use polars::prelude::*;

fn main() -> PolarsResult<()> {
    let rental_lazy = LazyCsvReader::new("Rent_Contracts.csv".into())
        .with_has_header(true) // Explicitly tell Polars there is a header row
        .finish()?;

    // Data isn't actually processed until after query optimization.
    // Rust then executes the operations, generating a DataFrame 
    // as a result:
    //
    // let df = rental_lazy.collect()?;

    Ok(())
}

Whereas the Python API throws runtime exceptions on failure, Rust methods return PolarsResult<T>. Using the Rust ? operator allows us to structure our code to propagate errors and handle them safely.

Display First Rows of a DataFrame

By default, the Python method head() displays the first five rows of a DataFrame. We can also specify the number of rows we want: rentals.head(3)

import polars as pl

rental = pl.read_csv("Rent_Contracts.csv")
print(rental.head(3))

For Rust LazyFrame, we use the limit() method instead.

use polars::prelude::*;

fn main() -> PolarsResult<()> {
    // 1. Initialize the LazyCsvReader with explicit options
    let rental_lazy = LazyCsvReader::new("Rent_Contracts.csv".into())
        .with_has_header(true) // Explicitly tell Polars there is a header row
        .finish()?;

    // 2. Limit to 3 rows to optimize the query plan, then collect to execute
    let df = rental_lazy.limit(3).collect()?;

    // 3. Display the results
    println!("{}", df);

    Ok(())
}

shape: (3, 30)
┌───────────────┬──────────────────┬──────────────────┬─────────────────┬───┬────────────────┬────────────────┬─────────────────┬──────────┐
│ contract_id   ┆ contract_reg_typ ┆ contract_reg_typ ┆ contract_start_ ┆ … ┆ tenant_type_id ┆ tenant_type_en ┆ project_name    ┆ rooms_en │
│ ---           ┆ e_id             ┆ e_en             ┆ date            ┆   ┆ ---            ┆ ---            ┆ ---             ┆ ---      │
│ str           ┆ ---              ┆ ---              ┆ ---             ┆   ┆ f64            ┆ str            ┆ str             ┆ str      │
│               ┆ i64              ┆ str              ┆ str             ┆   ┆                ┆                ┆                 ┆          │
╞═══════════════╪══════════════════╪══════════════════╪═════════════════╪═══╪════════════════╪════════════════╪═════════════════╪══════════╡
│ CRT2128114436 ┆ 1                ┆ New              ┆ 2025-12-31      ┆ … ┆ 1.0            ┆ Person         ┆ Azizi Riviera   ┆ Studio   │
│               ┆                  ┆                  ┆                 ┆   ┆                ┆                ┆ 35              ┆          │
│ CNT1786418853 ┆ 2                ┆ Renew            ┆ 2025-12-31      ┆ … ┆ 1.0            ┆ Person         ┆ Discovery       ┆ 1 B/R    │
│               ┆                  ┆                  ┆                 ┆   ┆                ┆                ┆ Gardens         ┆          │
│ CNT2126820013 ┆ 2                ┆ Renew            ┆ 2025-12-31      ┆ … ┆ 1.0            ┆ Person         ┆ Meydan          ┆ 2 B/R    │
└───────────────┴──────────────────┴──────────────────┴─────────────────┴───┴────────────────┴────────────────┴─────────────────┴──────────┘

The shape of the LazyFrame is printed at the very top of the table, telling us there 3 rows and 30 columns in the displayed output. Each column is capped by a column name and the column data type (dtype). As the example indicates, data columns have varying data types, ranging from string to 64-bit signed inetegers or 64-bit floating point numbers.

We can also use the tail() method to get the last rows of the Lazyframe.

use polars::prelude::*;

fn main() -> PolarsResult<()> {
    // 1. Initialize the LazyCsvReader with explicit options
    let rental_lazy = LazyCsvReader::new("Rent_Contracts.csv".into())
        .with_has_header(true) // Explicitly tell Polars there is a header row
        .finish()?;

    // 2. Limit to the last 3 rows to optimize the query plan, then collect
    let df = rental_lazy.tail(3).collect()?;

    // 3. Display the results
    println!("{}", df);

    Ok(())
}

shape: (3, 30)
┌───────────────┬───────────────────┬───────────────────┬──────────────────┬───┬────────────────┬────────────────┬──────────────┬──────────┐
│ contract_id   ┆ contract_reg_type ┆ contract_reg_type ┆ contract_start_d ┆ … ┆ tenant_type_id ┆ tenant_type_en ┆ project_name ┆ rooms_en │
│ ---           ┆ _id               ┆ _en               ┆ ate              ┆   ┆ ---            ┆ ---            ┆ ---          ┆ ---      │
│ str           ┆ ---               ┆ ---               ┆ ---              ┆   ┆ f64            ┆ str            ┆ str          ┆ str      │
│               ┆ i64               ┆ str               ┆ str              ┆   ┆                ┆                ┆              ┆          │
╞═══════════════╪═══════════════════╪═══════════════════╪══════════════════╪═══╪════════════════╪════════════════╪══════════════╪══════════╡
│ CNT208147537  ┆ 1                 ┆ New               ┆ 2014-02-01       ┆ … ┆ 2.0            ┆ Authority      ┆ null         ┆ None B/R │
│ CRT1182225816 ┆ 1                 ┆ New               ┆ 2013-06-20       ┆ … ┆ 1.0            ┆ Person         ┆ GRANDEUR     ┆ 4 B/R    │
│               ┆                   ┆                   ┆                  ┆   ┆                ┆                ┆ RESIDENCES   ┆          │
│ CNT152225829  ┆ 2                 ┆ Renew             ┆ 2013-02-25       ┆ … ┆ 1.0            ┆ Person         ┆ null         ┆ 1 B/R    │
└───────────────┴───────────────────┴───────────────────┴──────────────────┴───┴────────────────┴────────────────┴──────────────┴──────────┘

The Python glimpse() method displays schema information. When we are dealing with a LazyFrame in Rust, we use the collect_schema() method.

use polars::prelude::*;

fn main() -> PolarsResult<()> {
    let mut rental_lazy = LazyCsvReader::new("Rent_Contracts.csv".into())
        .with_has_header(true)
        .finish()?;

    // Fetch the schema (column names and data types) without executing the full query
    let schema = rental_lazy.collect_schema()?;

    // Use the {:#?} pretty-print debug formatter to make it readable
    println!("{:#?}", schema);

    Ok(())
}

Schema {
    fields: {
        "contract_id": String,
        "contract_reg_type_id": Int64,
        "contract_reg_type_en": String,
        "contract_start_date": String,
        "contract_end_date": String,
        "contract_amount": Int64,
        "annual_amount": Int64,
        "no_of_prop": Int64,
        "line_number": Int64,
        "is_free_hold": Float64,
        "ejari_bus_property_type_id": Float64,
        "ejari_bus_property_type_en": String,
        "ejari_property_type_id": Float64,
        "ejari_property_type_en": String,
        "ejari_property_sub_type_id": String,
        "ejari_property_sub_type_en": String,
        "property_usage_en": String,
        "project_number": Float64,
        "project_name_en": String,
        "master_project_en": String,
        "area_id": Float64,
        "area_name_en": String,
        "actual_area": Float64,
        "nearest_landmark_en": String,
        "nearest_metro_en": String,
        "nearest_mall_en": String,
        "tenant_type_id": Float64,
        "tenant_type_en": String,
        "project_name": String,
        "rooms_en": String,
    },
    metadata: (),
}

Introspecting a LazyFrame

To round out the discussion, here are some more code snippets for uncovering additional schema informaton.

1. Print number of rows and columns

import polars as pl

rental = pl.read_csv("Rent_Contracts.csv")
print(rental.shape)

// Execute the query to get an Eager DataFrame, then check the shape
let eager_df = lazy_frame.clone().collect()?; 
let (rows, cols) = eager_df.shape();
println!("Shape of DataFrame: ({}, {})", rows, cols);

Note that our Rust code we need to generate a DataFrame from a LazyFrame before we apply the shape() operation. Initially a LazyFrame represents a plan of execution (a query plan). The structure of the data is unknown at this point. It’s only with the call to collect() that Polars carries out the plan, yielding a DataFrame with set columns and rows.

2. Print the column names

import polars as pl

rental = pl.read_csv("Rent_Contracts.csv")
print(rental.columns)

Polars does know the column names of a LazyFrame prior to execution because it tracks them in the query plan’s schema. Thus, our Rust example below, we can fetch the schema and extract the names without processing the whole data stream.

// Fetch the schema lazily, then iterate over the column names
let schema = lazy_frame.collect_schema()?;
let columns: Vec<&str> = schema.iter_names().map(|s| s.as_str()).collect();

println!("Columns: {:?}", columns);

3. Print the column names and dtypes

import polars as pl

rental = pl.read_csv("Rent_Contracts.csv")
print(rental.schema)

// Evaluate the query plan to deduce the schema from the LazyFrame
let schema = lazy_frame.collect_schema()?;

// Use {:#?} for a beautifully formatted vertical output
println!("{:#?}", schema);

4. Print the first values in a vertical format

import polars as pl

rental = pl.read_csv("Rent_Contracts.csv")
print(rental.glimpse)

The glimpse() operation is a convenience function provided only by the Python API. Internally it relies on the Rust API to supply building blocks to construct the result.

// Limit the query to 5 rows, execute it, and print the resulting table
let df_preview = lazy_frame.limit(5).collect()?;

// Standard printing already produces a rich ASCII table with data types
println!("{}", df_preview);