Polars Primer 07: Filtering Columns

Polars

Python

Rust

Data Analytics

A look at Polars using Python and Rust

Author

Dennis Chua

Published

September 10, 2026

The Polars Way to Select Rows

All code snippets use Rent_Contracts.csv as input data, sourced from lemoninabag/Rentals · Datasets at Hugging Face.

In earlier chapters we discussed the slice() operation used to isolate rows in a DataFrame. This method, available in both Python and Rust APIs, relies on indices, either a particular index or a range, to extract the rows. Polars offers a more flexible way that uses predicates. These are boolean expressions that filter data rows, picking up the ones for which the predicate evaluates to True. We’ll introduce these techniques in this chapter, together with helper functions that extend the utility of Polars predicates.

The `filter()` Operation

A predicate is a condition that evaluates to True or False. When used with the filter() operator, Polars selects rows in the DataFrame that matches the condition.

import polars as pl

eager_df = pl.read_csv("Rent_Contracts.csv")
filter_df = eager_df.filter( pl.col("contract_start_date") < "2014-12-31" )

print(filter_df)

The Python API overloads comparison operators such as less-than (<). With Rust, we have to be explicit. In our example below we call the lt() operator to evaluate the logic. Within the predicate we convert the string literal "2014-12-31" to a Polars Expr object required by the Polars query engine. Thanks to the engine’s query optimization, Polars will only read and load into memory the rows of data with contract_start_date earlier than "2014-12-31".

use polars::prelude::*;

fn main() -> PolarsResult<()> {
    // 1. Initialize the computation graph lazily
    let lf = LazyCsvReader::new("Rent_Contracts.csv".into())
        .with_has_header(true)
        .finish()?;

    // 2. Apply the filter lazily and execute the graph
    let filter_df = lf
        .filter(col("contract_start_date").lt(lit("2014-12-31")))
        .collect()?;

    println!("{:?}", filter_df);

    Ok(())
}

shape: (12, 30)
┌───────────┬───────────┬───────────┬──────────┬───┬──────────┬──────────┬──────────┬──────────┐
│ contract_ ┆ contract_ ┆ contract_ ┆ contract ┆ … ┆ tenant_t ┆ tenant_t ┆ project_ ┆ rooms_en │
│ id        ┆ reg_type_ ┆ reg_type_ ┆ _start_d ┆   ┆ ype_id   ┆ ype_en   ┆ name     ┆ ---      │
│ ---       ┆ id        ┆ en        ┆ ate      ┆   ┆ ---      ┆ ---      ┆ ---      ┆ str      │
│ str       ┆ ---       ┆ ---       ┆ ---      ┆   ┆ f64      ┆ str      ┆ str      ┆          │
│           ┆ i64       ┆ str       ┆ str      ┆   ┆          ┆          ┆          ┆          │
╞═══════════╪═══════════╪═══════════╪══════════╪═══╪══════════╪══════════╪══════════╪══════════╡
│ CRT260094 ┆ 1         ┆ New       ┆ 2014-11- ┆ … ┆ null     ┆ null     ┆ ELITE 5  ┆ Studio   │
│ 846       ┆           ┆           ┆ 20       ┆   ┆          ┆          ┆ SPORTS   ┆          │
│           ┆           ┆           ┆          ┆   ┆          ┆          ┆ RESIDENC ┆          │
│           ┆           ┆           ┆          ┆   ┆          ┆          ┆ E        ┆          │
│ CRT228336 ┆ 1         ┆ New       ┆ 2014-06- ┆ … ┆ null     ┆ null     ┆ null     ┆ Studio   │
│ 726       ┆           ┆           ┆ 17       ┆   ┆          ┆          ┆          ┆          │
│ CNT837594 ┆ 1         ┆ New       ┆ 2014-05- ┆ … ┆ 2.0      ┆ Authorit ┆ null     ┆ 1 B/R    │
│ 775       ┆           ┆           ┆ 06       ┆   ┆          ┆ y        ┆          ┆          │
│ CNT807529 ┆ 1         ┆ New       ┆ 2014-05- ┆ … ┆ 2.0      ┆ Authorit ┆ null     ┆ 1 B/R    │
│ 343       ┆           ┆           ┆ 06       ┆   ┆          ┆ y        ┆          ┆          │
│ CNT693138 ┆ 1         ┆ New       ┆ 2014-05- ┆ … ┆ 2.0      ┆ Authorit ┆ null     ┆ 1 B/R    │
│ 643       ┆           ┆           ┆ 05       ┆   ┆          ┆ y        ┆          ┆          │
│ …         ┆ …         ┆ …         ┆ …        ┆ … ┆ …        ┆ …        ┆ …        ┆ …        │
│ CNT209080 ┆ 1         ┆ New       ┆ 2014-02- ┆ … ┆ 2.0      ┆ Authorit ┆ null     ┆ 9 B/R    │
│ 015       ┆           ┆           ┆ 01       ┆   ┆          ┆ y        ┆          ┆          │
│ CNT345228 ┆ 1         ┆ New       ┆ 2014-02- ┆ … ┆ 2.0      ┆ Authorit ┆ null     ┆ None B/R │
│ 436       ┆           ┆           ┆ 01       ┆   ┆          ┆ y        ┆          ┆          │
│ CNT208147 ┆ 1         ┆ New       ┆ 2014-02- ┆ … ┆ 2.0      ┆ Authorit ┆ null     ┆ None B/R │
│ 537       ┆           ┆           ┆ 01       ┆   ┆          ┆ y        ┆          ┆          │
│ CRT118222 ┆ 1         ┆ New       ┆ 2013-06- ┆ … ┆ 1.0      ┆ Person   ┆ GRANDEUR ┆ 4 B/R    │
│ 5816      ┆           ┆           ┆ 20       ┆   ┆          ┆          ┆ RESIDENC ┆          │
│           ┆           ┆           ┆          ┆   ┆          ┆          ┆ ES       ┆          │
│ CNT152225 ┆ 2         ┆ Renew     ┆ 2013-02- ┆ … ┆ 1.0      ┆ Person   ┆ null     ┆ 1 B/R    │
│ 829       ┆           ┆           ┆ 25       ┆   ┆          ┆          ┆          ┆          │
└───────────┴───────────┴───────────┴──────────┴───┴──────────┴──────────┴──────────┴──────────┘

Polars accepts more than one column in the predicate expression. We simply string these together using logical operators.

Sticking to our first example, we update this to filter for rows with "tenant_type_id" equal to 1.0.

import polars as pl

eager_df = pl.read_csv("Rent_Contracts.csv")
filter_df = eager_df.filter( (pl.col("contract_start_date") < "2014-12-31") & 
                             (pl.col("tenant_type_id") == 1.0))

print(filter_df)

Once again, Rust requires us to explicitly call logic functions, in this case and() and eq() and lt().

use polars::prelude::*;

fn main() -> PolarsResult<()> {
    let lf = LazyCsvReader::new("Rent_Contracts.csv".into())
        .with_has_header(true)
        .finish()?;

    let filter_df = lf
        .filter(
            col("contract_start_date").lt(lit("2014-12-31"))
            .and(col("tenant_type_id").eq(lit(1.0)))
        )
        .collect()?;

    println!("{:?}", filter_df);

    Ok(())
}

shape: (2, 30)
┌───────────┬───────────┬───────────┬──────────┬───┬──────────┬──────────┬──────────┬──────────┐
│ contract_ ┆ contract_ ┆ contract_ ┆ contract ┆ … ┆ tenant_t ┆ tenant_t ┆ project_ ┆ rooms_en │
│ id        ┆ reg_type_ ┆ reg_type_ ┆ _start_d ┆   ┆ ype_id   ┆ ype_en   ┆ name     ┆ ---      │
│ ---       ┆ id        ┆ en        ┆ ate      ┆   ┆ ---      ┆ ---      ┆ ---      ┆ str      │
│ str       ┆ ---       ┆ ---       ┆ ---      ┆   ┆ f64      ┆ str      ┆ str      ┆          │
│           ┆ i64       ┆ str       ┆ str      ┆   ┆          ┆          ┆          ┆          │
╞═══════════╪═══════════╪═══════════╪══════════╪═══╪══════════╪══════════╪══════════╪══════════╡
│ CRT118222 ┆ 1         ┆ New       ┆ 2013-06- ┆ … ┆ 1.0      ┆ Person   ┆ GRANDEUR ┆ 4 B/R    │
│ 5816      ┆           ┆           ┆ 20       ┆   ┆          ┆          ┆ RESIDENC ┆          │
│           ┆           ┆           ┆          ┆   ┆          ┆          ┆ ES       ┆          │
│ CNT152225 ┆ 2         ┆ Renew     ┆ 2013-02- ┆ … ┆ 1.0      ┆ Person   ┆ null     ┆ 1 B/R    │
│ 829       ┆           ┆           ┆ 25       ┆   ┆          ┆          ┆          ┆          │
└───────────┴───────────┴───────────┴──────────┴───┴──────────┴──────────┴──────────┴──────────┘

Filtering Rows According to Enumerations

In case we have a predetermined collection of values on hand to select rows, Polars has set operations that match for membership. For example, this Python snippet lists all the unique values in the "nearest_mall_en" column.

import polars as pl

eager_df = pl.read_csv("Rent_Contracts.csv")
list_malls = eager_df["nearest_malls_en"].unique().to_list()

print(list_malls)

The Rust version is more detailed. At its core it uses the same logic as our Python script, but we take added steps to convert the Series data into a vector of strings suitable for console display.

use polars::prelude::*;

fn main() -> PolarsResult<()> {
    // 1. Initialize the computation graph lazily
    let lf = LazyCsvReader::new("Rent_Contracts.csv".into())
        .with_has_header(true)
        .finish()?;

    // 2. Select the column, apply the unique expression lazily, and collect
    let unique_df = lf
        .select([col("nearest_mall_en").unique()])
        .collect()?;

    // 3. Extract the Series and convert it into a native Rust Vector
    // We downcast to a string array (.str()?) and use .iter() instead of .into_iter()
    let list_malls: Vec<Option<&str>> = unique_df
        .column("nearest_mall_en")?
        .str()?
        .iter()
        .collect();

    println!("{:?}", list_malls);

    Ok(())
}

['Dubai Mall', 'Ibn-e-Battuta Mall', 'City Centre Mirdif', 'Mall of the Emirates', None, 'Marina Mall']

We can pick a handful of values from this list of shopping malls and select the rows that match using the is_in() operator. For the sake of clarity, we select() only two output columns.

more polars_test.py 
import polars as pl

eager_df = pl.read_csv("Rent_Contracts.csv")
subset_df = eager_df.select("contract_id", "nearest_mall_en").filter(
            pl.col("nearest_mall_en").is_in(['Dubai Mall', 'Ibn-e-Battuta Mall']))

print(subset_df)

The Rust equivalent is nearly the same, except that we need to use a Series type to encapuslate that select list of shopping malls.

use polars::prelude::*;

fn main() -> PolarsResult<()> {
    // 1. Initialize the computation graph lazily
    let lf = LazyCsvReader::new("Rent_Contracts.csv".into())
        .with_has_header(true)
        .finish()?;

    // 2. Construct the target elements as a Polars Series
    let target_malls = Series::new("malls".into(), &["Dubai Mall", "Ibn-e-Battuta Mall"]);

    // 3. Select columns, apply the is_in filter with implode(false) and the nulls_equal flag
    let subset_df = lf
        .select([col("contract_id"), col("nearest_mall_en")])
        .filter(
            // Pass 'false' to implode() to ignore sorting overhead
            col("nearest_mall_en").is_in(lit(target_malls).implode(false), false)
        )
        .collect()?;

    println!("{:?}", subset_df);

    Ok(())
}

shape: (426_748, 2)
┌───────────────┬────────────────────┐
│ contract_id   ┆ nearest_mall_en    │
│ ---           ┆ ---                │
│ str           ┆ str                │
╞═══════════════╪════════════════════╡
│ CRT2128114436 ┆ Dubai Mall         │
│ CNT1786418853 ┆ Ibn-e-Battuta Mall │
│ CNT2126820013 ┆ Dubai Mall         │
│ CNT2128949309 ┆ Dubai Mall         │
│ CNT2126854343 ┆ Dubai Mall         │
│ …             ┆ …                  │
│ CNT807529343  ┆ Dubai Mall         │
│ CNT693138643  ┆ Dubai Mall         │
│ CNT689140544  ┆ Dubai Mall         │
│ CNT258538347  ┆ Dubai Mall         │
│ CNT152225829  ┆ Dubai Mall         │
└───────────────┴────────────────────┘

In the Rust version, the expression involving is_in() incorporates lit() and implode(). Let’s discuss the purpose each one.

Recall that the Polars lazy query plan deals with Expr objects. After we encapsulated the list of shopping mall names within a Series object, target_malls, we call lit(target_malls) to cast this column of data as a constant Expr for the query engine to use.
In an earlier chapter we discussed the list namespace. In that context, the implode() operator we used here compresses a Series into a List data type. This facilitates set membership logic, testing whether the value in each cell of the "nearest_mall_en" column appears in the List of shopping mall names.

The false value maps to the maintain_orderparameter of implode() and optimizes the set membership operation. Recall that Polars relies on the Rayon threading engine to split a column into chunks for parallel processing. When translating a Series into a List, Polars tracks not just the elements themselves but also their location indicated by their ordering. This incurs an overhead, requiring Polars to shedule the Rayon threads to maintain that order. By setting the implode()parameter to false, we instruct Polars to overlook the sorting of the string elements, saving it the extra CPU processing needed to coordinate the Rayon threads.

The last point we will make about is_in() is that chaining not_()reverses the logic of the membership test. When we use the Python expression is_in(['Dubai Mall', 'Ibn-e-Battuta Mall']).not_(), we are filtering for rows with "nearest_mall_en" values that are anything but those two mall names we’ve enumerated.

The Rust API is similar:

let subset_df = lf
        .select([col("contract_id"), col("nearest_mall_en")])
        .filter(
            col("nearest_mall_en")
                .is_in(lit(target_malls).implode(false), false)
                .not() // Inverts the match to EXCLUDE these malls
        )
        .collect()?;

The Polars Way to Select Rows

The filter() Operation

Filtering Rows According to Enumerations

The `filter()` Operation