index.html

<!DOCTYPE html>
<html lang="en">
<head>
    <!--
        Developer: Molla Samser
        Designer & Tester: Rima Khatun
        Website: https://rskworld.in
        Email: help@rskworld.in, support@rskworld.in, info@rskworld.com
        Phone: +91 93305 39277
        Company: RSK World
        Description: Fraud Detection Dataset - Demo Page
    -->
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Fraud Detection Dataset - RSK World</title>
    <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/css/bootstrap.min.css" rel="stylesheet">
    <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/6.4.0/css/all.min.css">
    <style>
        /* Developer: Molla Samser | Website: https://rskworld.in */
        body {
            font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
            min-height: 100vh;
            padding: 20px 0;
        }
        .container {
            max-width: 1200px;
        }
        .header-card {
            background: white;
            border-radius: 15px;
            box-shadow: 0 10px 30px rgba(0,0,0,0.2);
            padding: 30px;
            margin-bottom: 30px;
        }
        .header-card h1 {
            color: #667eea;
            font-weight: bold;
        }
        .feature-card {
            background: white;
            border-radius: 15px;
            box-shadow: 0 5px 15px rgba(0,0,0,0.1);
            padding: 25px;
            margin-bottom: 20px;
            transition: transform 0.3s ease;
        }
        .feature-card:hover {
            transform: translateY(-5px);
            box-shadow: 0 8px 25px rgba(0,0,0,0.15);
        }
        .feature-card i {
            font-size: 2.5rem;
            color: #667eea;
            margin-bottom: 15px;
        }
        .stats-card {
            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
            color: white;
            border-radius: 15px;
            padding: 25px;
            margin-bottom: 20px;
            text-align: center;
        }
        .stats-card h3 {
            font-size: 2.5rem;
            font-weight: bold;
            margin-bottom: 10px;
        }
        .btn-custom {
            background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
            border: none;
            color: white;
            padding: 12px 30px;
            border-radius: 25px;
            font-weight: bold;
            transition: all 0.3s ease;
        }
        .btn-custom:hover {
            transform: scale(1.05);
            box-shadow: 0 5px 15px rgba(102, 126, 234, 0.4);
            color: white;
        }
        .tech-badge {
            display: inline-block;
            background: #e9ecef;
            padding: 8px 15px;
            border-radius: 20px;
            margin: 5px;
            font-size: 0.9rem;
        }
        .footer {
            background: white;
            border-radius: 15px;
            padding: 30px;
            margin-top: 30px;
            text-align: center;
        }
        .footer a {
            color: #667eea;
            text-decoration: none;
        }
        .footer a:hover {
            text-decoration: underline;
        }
    </style>
</head>
<body>
    <div class="container">
        <!-- Header -->
        <div class="header-card">
            <div class="text-center">
                <h1><i class="fas fa-shield-alt text-warning"></i> Fraud Detection Dataset</h1>
                <p class="lead mt-3">Financial fraud detection dataset with transaction records, user behavior patterns, and fraud labels for building anti-fraud ML models.</p>
                <div class="mt-4">
                    <span class="badge bg-primary me-2">Tabular Data</span>
                    <span class="badge bg-warning text-dark">Intermediate</span>
                    <span class="badge bg-success">Machine Learning</span>
                </div>
            </div>
        </div>

        <!-- Statistics -->
        <div class="row">
            <div class="col-md-3">
                <div class="stats-card">
                    <h3>10,000</h3>
                    <p class="mb-0">Transactions</p>
                </div>
            </div>
            <div class="col-md-3">
                <div class="stats-card">
                    <h3>15+</h3>
                    <p class="mb-0">Features</p>
                </div>
            </div>
            <div class="col-md-3">
                <div class="stats-card">
                    <h3>5%</h3>
                    <p class="mb-0">Fraud Ratio</p>
                </div>
            </div>
            <div class="col-md-3">
                <div class="stats-card">
                    <h3>100%</h3>
                    <p class="mb-0">Ready to Use</p>
                </div>
            </div>
        </div>

        <!-- Description -->
        <div class="header-card">
            <h2><i class="fas fa-info-circle text-primary"></i> Description</h2>
            <p class="mt-3">This dataset contains transaction records with features like transaction amount, location, time, merchant information, and fraud labels. Perfect for building fraud detection models, anomaly detection, and financial security applications.</p>
        </div>

        <!-- Features -->
        <div class="row">
            <div class="col-md-6">
                <div class="feature-card">
                    <i class="fas fa-list-alt"></i>
                    <h4>Transaction Records</h4>
                    <p>Comprehensive transaction data with detailed information including amounts, timestamps, and merchant details.</p>
                </div>
            </div>
            <div class="col-md-6">
                <div class="feature-card">
                    <i class="fas fa-user-chart"></i>
                    <h4>User Behavior Features</h4>
                    <p>Features capturing user behavior patterns including transaction frequency, account age, and transaction history.</p>
                </div>
            </div>
            <div class="col-md-6">
                <div class="feature-card">
                    <i class="fas fa-flag"></i>
                    <h4>Fraud Labels</h4>
                    <p>Binary labels (fraud/normal) for supervised learning and model evaluation.</p>
                </div>
            </div>
            <div class="col-md-6">
                <div class="feature-card">
                    <i class="fas fa-balance-scale"></i>
                    <h4>Imbalanced Dataset</h4>
                    <p>Realistic imbalanced dataset (5% fraud) for practicing imbalanced classification techniques.</p>
                </div>
            </div>
            <div class="col-md-12">
                <div class="feature-card">
                    <i class="fas fa-robot"></i>
                    <h4>Ready for Classification Models</h4>
                    <p>Preprocessed and structured data ready for machine learning models including Random Forest, XGBoost, and Neural Networks.</p>
                </div>
            </div>
        </div>

        <!-- Dataset Features -->
        <div class="header-card">
            <h2><i class="fas fa-table text-success"></i> Dataset Features</h2>
            <div class="row mt-4">
                <div class="col-md-6">
                    <ul class="list-group">
                        <li class="list-group-item"><i class="fas fa-check-circle text-success me-2"></i>transaction_id</li>
                        <li class="list-group-item"><i class="fas fa-check-circle text-success me-2"></i>user_id</li>
                        <li class="list-group-item"><i class="fas fa-check-circle text-success me-2"></i>amount</li>
                        <li class="list-group-item"><i class="fas fa-check-circle text-success me-2"></i>merchant_category</li>
                        <li class="list-group-item"><i class="fas fa-check-circle text-success me-2"></i>location</li>
                        <li class="list-group-item"><i class="fas fa-check-circle text-success me-2"></i>timestamp</li>
                        <li class="list-group-item"><i class="fas fa-check-circle text-success me-2"></i>device_type</li>
                    </ul>
                </div>
                <div class="col-md-6">
                    <ul class="list-group">
                        <li class="list-group-item"><i class="fas fa-check-circle text-success me-2"></i>user_age</li>
                        <li class="list-group-item"><i class="fas fa-check-circle text-success me-2"></i>account_age_days</li>
                        <li class="list-group-item"><i class="fas fa-check-circle text-success me-2"></i>transaction_count_24h</li>
                        <li class="list-group-item"><i class="fas fa-check-circle text-success me-2"></i>avg_transaction_amount</li>
                        <li class="list-group-item"><i class="fas fa-check-circle text-success me-2"></i>is_foreign_transaction</li>
                        <li class="list-group-item"><i class="fas fa-check-circle text-success me-2"></i>is_weekend</li>
                        <li class="list-group-item"><i class="fas fa-check-circle text-success me-2"></i>hour_of_day</li>
                        <li class="list-group-item"><i class="fas fa-check-circle text-success me-2"></i>is_fraud (target)</li>
                    </ul>
                </div>
            </div>
        </div>

        <!-- Technologies -->
        <div class="header-card">
            <h2><i class="fas fa-code text-info"></i> Technologies</h2>
            <div class="mt-3">
                <span class="tech-badge"><i class="fas fa-file-csv"></i> CSV</span>
                <span class="tech-badge"><i class="fas fa-file-excel"></i> Excel</span>
                <span class="tech-badge"><i class="fab fa-python"></i> Pandas</span>
                <span class="tech-badge"><i class="fas fa-chart-line"></i> Scikit-learn</span>
                <span class="tech-badge"><i class="fab fa-python"></i> NumPy</span>
                <span class="tech-badge"><i class="fas fa-chart-bar"></i> Matplotlib</span>
                <span class="tech-badge"><i class="fas fa-project-diagram"></i> Jupyter</span>
            </div>
        </div>

        <!-- Download & Usage -->
        <div class="header-card text-center">
            <h2><i class="fas fa-download text-danger"></i> Get Started</h2>
            <p class="mt-3">Download the dataset and start building fraud detection models</p>
            <div class="mt-4">
                <a href="fraud_detection_dataset.csv" class="btn btn-custom me-3" download>
                    <i class="fas fa-download"></i> Download CSV
                </a>
                <a href="README.md" class="btn btn-outline-primary me-3">
                    <i class="fas fa-book"></i> Read Documentation
                </a>
                <a href="fraud_detection_analysis.ipynb" class="btn btn-outline-success">
                    <i class="fas fa-code"></i> View Notebook
                </a>
            </div>
        </div>

        <!-- Footer -->
        <div class="footer">
            <h5>RSK World</h5>
            <p class="mb-2">Free Programming Resources & Source Code</p>
            <p class="mb-2">
                <i class="fas fa-globe me-2"></i>
                <a href="https://rskworld.in" target="_blank">https://rskworld.in</a>
            </p>
            <p class="mb-2">
                <i class="fas fa-envelope me-2"></i>
                <a href="mailto:help@rskworld.in">help@rskworld.in</a> | 
                <a href="mailto:support@rskworld.in">support@rskworld.in</a>
            </p>
            <p class="mb-0">
                <i class="fas fa-phone me-2"></i>
                <a href="tel:+919330539277">+91 93305 39277</a>
            </p>
            <hr class="my-3">
            <p class="text-muted small mb-0">
                Developer: Molla Samser | Designer & Tester: Rima Khatun<br>
                Content used for educational purposes only.
            </p>
        </div>
    </div>

    <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/js/bootstrap.bundle.min.js"></script>
</body>
</html>

286 lines•12.3 KB

markup

README.md

Raw Download

README.md

# Fraud Detection Dataset



Financial fraud detection dataset with transaction records, user behavior patterns, and fraud labels for building anti-fraud ML models.

## Description

This dataset contains transaction records with features like transaction amount, location, time, merchant information, and fraud labels. Perfect for building fraud detection models, anomaly detection, and financial security applications.

## Features

- **Transaction records** - Comprehensive transaction data
- **User behavior features** - Advanced behavioral patterns
- **Fraud labels** - Binary classification labels
- **Imbalanced dataset handling** - Realistic 5% fraud ratio
- **Ready for classification models** - Preprocessed and structured
- **Advanced feature engineering** - Time-based, statistical, and interaction features
- **Multiple ML models** - Random Forest, XGBoost, LightGBM support

## Technologies

- CSV, Excel
- Pandas, NumPy
- Scikit-learn, XGBoost, LightGBM
- SHAP (for model explainability)
- SMOTE (for imbalanced data handling)

## Dataset Structure

### Basic Features
- `transaction_id`: Unique identifier for each transaction
- `user_id`: User identifier
- `amount`: Transaction amount
- `merchant_category`: Category of the merchant
- `location`: Transaction location
- `timestamp`: Transaction timestamp
- `device_type`: Device used for transaction
- `user_age`: Age of the user
- `account_age_days`: Age of the account in days
- `transaction_count_24h`: Number of transactions in last 24 hours
- `avg_transaction_amount`: Average transaction amount
- `is_foreign_transaction`: Whether transaction is from foreign location
- `is_weekend`: Whether transaction occurred on weekend
- `hour_of_day`: Hour of the day (0-23)
- `is_fraud`: Fraud label (0 = Normal, 1 = Fraud)

### Advanced Features (Auto-generated)
- **Time-based features**: day_of_week, month, is_month_end, is_month_start, quarter
- **Transaction velocity**: time_since_last_transaction, transactions_per_hour, transactions_per_day
- **Statistical features**: amount_zscore, amount_percentile, amount_deviation_from_avg
- **Rolling statistics**: amount_rolling_mean_7d, amount_rolling_std_7d, amount_rolling_max_7d
- **Change indicators**: location_changed, device_changed, merchant_category_changed
- **Risk scoring**: risk_score (composite risk indicator)
- **Interaction features**: amount_x_transaction_count, amount_x_is_foreign, etc.
- **Time patterns**: is_rush_hour, is_off_hours, is_business_hours
- **Behavioral patterns**: avg_amount_ratio, transaction_velocity
- **Account features**: account_age_months, is_new_account, is_mature_account

## Installation

```bash
pip install -r requirements.txt
```

## Usage

### 1. Generate the dataset:
```bash
python generate_data.py
```

### 2. Engineer advanced features (optional):
```bash
python advanced_feature_engineering.py
```

### 3. Explore the data:
```bash
jupyter notebook fraud_detection_analysis.ipynb
```

### 4. Train models:
```bash
# Basic training
python train_model.py

# The script will automatically:
# - Use advanced feature engineering
# - Train multiple models (Random Forest, XGBoost, LightGBM)
# - Compare model performance
# - Generate evaluation plots
```

## Advanced Features

### 1. Advanced Feature Engineering (`advanced_feature_engineering.py`)
Comprehensive feature engineering module that creates:
- **Time-based features**: day_of_week, month, quarter, is_month_end, is_month_start
- **Time patterns**: is_rush_hour, is_off_hours, is_business_hours
- **Statistical features**: amount_zscore, amount_percentile, amount_deviation_from_avg
- **Rolling statistics**: amount_rolling_mean_7d, amount_rolling_std_7d, amount_rolling_max_7d
- **Transaction velocity**: transaction_velocity, is_high_velocity
- **Change indicators**: location_changed, device_changed
- **Risk scoring**: risk_score (composite risk indicator)
- **Interaction features**: amount_x_transaction_count, amount_x_is_foreign, amount_x_hour
- **Account features**: account_age_months, is_new_account, is_mature_account
- **User behavior patterns**: user_avg_amount, user_std_amount, amount_deviation_from_user_avg

### 2. Hyperparameter Tuning (`hyperparameter_tuning.py`)
- Automated hyperparameter optimization using RandomizedSearchCV
- Supports Random Forest, XGBoost, and LightGBM
- Cross-validation for robust parameter selection
- Automatic best model selection and saving
- Results exported to CSV for analysis

### 3. Advanced Model Evaluation (`model_evaluation_advanced.py`)
Comprehensive evaluation with:
- Multiple metrics: Accuracy, Precision, Recall, F1, ROC-AUC, Average Precision
- Additional metrics: Log Loss, Matthews Correlation Coefficient, Cohen's Kappa
- Cross-validation with multiple scoring metrics
- ROC curve comparison across models
- Precision-Recall curve analysis
- Feature importance comparison
- Metrics visualization and comparison plots

### 4. Model Explainability (`shap_explainability.py`)
SHAP (SHapley Additive exPlanations) integration:
- SHAP summary plots
- SHAP waterfall plots for individual predictions
- Feature importance from SHAP values
- Model interpretation and explainability

### 5. Prediction Pipeline (`predict_pipeline.py`)
Production-ready prediction system:
- Batch prediction from CSV files
- Single transaction prediction
- Risk level classification (Low/Medium/High)
- Probability scores
- Ready for deployment

### 6. Model Training (`train_model.py`)
Training script with:
- **Random Forest**: Robust ensemble method with balanced class weights
- **XGBoost**: Gradient boosting with advanced regularization
- **LightGBM**: Fast gradient boosting framework
- Automatic model comparison and best model selection
- SMOTE for handling imbalanced data
- Comprehensive evaluation metrics

## Difficulty Level

Intermediate to Advanced

## Project Structure

```
fraud-detection/
├── fraud_detection_dataset.csv # Generated dataset
├── generate_data.py # Dataset generation script
├── advanced_feature_engineering.py # Advanced feature engineering
├── train_model.py # Model training script
├── hyperparameter_tuning.py # Hyperparameter optimization
├── model_evaluation_advanced.py # Advanced evaluation metrics
├── shap_explainability.py # SHAP model explainability
├── predict_pipeline.py # Production prediction pipeline
├── fraud_detection_analysis.ipynb # Jupyter notebook for analysis
├── verify_dataset.py # Dataset verification script
├── requirements.txt # Python dependencies
├── README.md # This file
├── index.html # Demo page
└── .gitignore # Git ignore file
```

## Model Outputs

After training and evaluation, the following files may be generated:
- `fraud_detection_model.pkl` - Best trained model
- `best_tuned_model_*.pkl` - Best tuned model from hyperparameter optimization
- `confusion_matrix.png` - Confusion matrix visualization
- `roc_curve.png` - ROC curve comparison
- `feature_importance.png` - Feature importance plot
- `hyperparameter_tuning_results.csv` - Hyperparameter tuning results
- `fraud_detection_dataset_advanced.csv` - Dataset with advanced features
- `shap_summary.png` - SHAP summary plot (if SHAP is used)
- `shap_summary_bar.png` - SHAP summary bar plot

## Contact

For questions or support, please contact:
- **Website**: https://rskworld.in
- **Email**: help@rskworld.in, support@rskworld.in
- **Phone**: +91 93305 39277

## License

Content used for educational purposes only.

## Credits

- **Developer**: Molla Samser
- **Designer & Tester**: Rima Khatun
- **Company**: RSK World

Theme Settings

Color Scheme

Display Options

Font Size

README.md