README.md

# Polars Fast DataFrames

<!--
Author: RSK World
Website: https://rskworld.in
Email: help@rskworld.in
Phone: +91 93305 39277
-->

High-performance DataFrame library with Polars for fast data processing, querying, and analysis on large datasets.

## Description

This project demonstrates Polars, a blazingly fast DataFrame library written in Rust. It covers DataFrame operations, lazy evaluation, query optimization, and performance comparisons. Perfect for high-performance data processing and analysis.

## Features

- ⚡ Fast DataFrame operations
- 🔄 Lazy evaluation and optimization
- 💾 Memory-efficient processing
- 🎯 Query optimization
- 🔗 Pandas compatibility
- 📊 Time series operations
- 🔍 Advanced string operations
- 🔗 Multiple join types (Inner, Left, Right, Outer, Anti, Semi)
- 📈 Advanced aggregations
- 🧹 Missing data handling
- 🔄 Data reshaping (Melt, Pivot)
- ✅ Data validation and quality checks
- 📦 Nested data structures (Struct, List)
- 🚀 Performance optimization techniques
- 💼 Real-world analytics examples

## Technologies

- Python
- Polars
- Pandas
- Jupyter Notebook

## Installation

```bash
pip install -r requirements.txt
```

## Project Structure

```
polars-fastdataframes/
├── README.md
├── requirements.txt
├── notebooks/
│ ├── 01_basic_operations.ipynb
│ ├── 02_lazy_evaluation.ipynb
│ ├── 03_performance_comparison.ipynb
│ └── 04_advanced_queries.ipynb
├── scripts/
│ ├── basic_operations.py
│ ├── lazy_evaluation.py
│ ├── performance_comparison.py
│ ├── advanced_queries.py
│ └── data_generator.py
└── data/
└── sample_data.csv
```

## Usage

### Jupyter Notebooks

Launch Jupyter Notebook and open any notebook from the `notebooks/` directory:

```bash
jupyter notebook
```

**Notebooks included:**
- `01_basic_operations.ipynb` - Basic DataFrame operations
- `02_lazy_evaluation.ipynb` - Lazy evaluation and query optimization
- `03_performance_comparison.ipynb` - Polars vs Pandas benchmarks
- `04_advanced_queries.ipynb` - **20+ advanced features** including:
- Time series operations
- Missing data handling
- Advanced string operations
- All join types
- Data validation
- Nested data structures
- Real-world analytics
- Performance optimization

### Python Scripts

Run any script from the `scripts/` directory:

```bash
# Basic operations
python scripts/basic_operations.py

# Lazy evaluation
python scripts/lazy_evaluation.py

# Performance comparison
python scripts/performance_comparison.py

# Advanced queries (NEW!)
python scripts/advanced_queries.py

# Generate sample data
python scripts/data_generator.py
```

## Performance

Polars is designed for speed and efficiency. It uses:
- Apache Arrow columnar memory format
- Query optimization through lazy evaluation
- Parallel processing capabilities
- Zero-copy reads

## License

This project is provided as educational material by RSK World.

## Contact

- Website: https://rskworld.in
- Email: help@rskworld.in
- Phone: +91 93305 39277