Refactor and document code; add new files
Refactored `script.py` by adding detailed docstrings and organizing functions. Created `.idea` configuration files and `gotodashboard.js` for `sisa_crawl` project. Added `readme.md` files with usage instructions and context for multiple scripts, and set up `package.json` for `sisa_crawl` dependencies.
This commit is contained in:
bdaneels
105
examengegevens template generator/readme.md
Normal file
105
examengegevens template generator/readme.md
Normal file
@@ -0,0 +1,105 @@
|
||||
# Project Name: Examination Data Processing
|
||||
|
||||
## Overview
|
||||
This project is designed to process examination data from an Excel file and generate filtered output and communication messages for teaching staff. It's developed using Python and pandas, and it provides functionalities such as filtering records, converting time formats, and generating message columns.
|
||||
|
||||
## Features
|
||||
- **Read Excel File**: Reads examination data from an Excel file into a Pandas DataFrame.
|
||||
- **Filter Data**: Filters records based on specific criteria in 'Studiegidsnummer' and 'Opmerkingen' columns.
|
||||
- **Convert Time Format**: Converts time columns to 'HH:MM' format.
|
||||
- **Generate Messages**: Creates message and subject columns for email communication.
|
||||
- **Save to Excel**: Saves the processed data to a new Excel file.
|
||||
|
||||
## Prerequisites
|
||||
- Python 3.12.5
|
||||
- Pandas
|
||||
- openpyxl
|
||||
|
||||
## Installation
|
||||
1. **Clone the repository**:
|
||||
```sh
|
||||
git clone https://github.com/username/examination-data-processing.git
|
||||
cd examination-data-processing
|
||||
```
|
||||
2. **Install the required Python packages**:
|
||||
```sh
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
Ensure the `requirements.txt` file should contain:
|
||||
```text
|
||||
pandas
|
||||
openpyxl
|
||||
```
|
||||
|
||||
## Usage
|
||||
1. **Place the input Excel file**: Ensure that the Excel file (`examengegevens2425.xlsx`) is placed in the root directory of the project.
|
||||
|
||||
2. **Run the script**:
|
||||
```sh
|
||||
python script.py
|
||||
```
|
||||
|
||||
3. **Output**: The filtered and processed data will be saved in an output Excel file (`filtered_examengegevens2425.xlsx`).
|
||||
|
||||
## Functions
|
||||
|
||||
### `read_excel_file(file_path)`
|
||||
- **Parameters**: `file_path` (str) - Path to the Excel file.
|
||||
- **Returns**: DataFrame or None
|
||||
|
||||
### `filter_studiegidsnummer(df)`
|
||||
- **Parameters**: `df` (DataFrame) - Input DataFrame.
|
||||
- **Returns**: Filtered DataFrame or empty DataFrame
|
||||
|
||||
### `filter_opmerkingen(df)`
|
||||
- **Parameters**: `df` (DataFrame) - Input DataFrame.
|
||||
- **Returns**: Filtered DataFrame or empty DataFrame
|
||||
|
||||
### `create_message_column(df)`
|
||||
- **Parameters**: `df` (DataFrame) - Input DataFrame.
|
||||
- **Returns**: DataFrame with 'Message' and 'subject' columns
|
||||
|
||||
### `save_to_excel(df, output_file_path)`
|
||||
- **Parameters**:
|
||||
- `df` (DataFrame) - DataFrame to save.
|
||||
- `output_file_path` (str) - Path to save the Excel file.
|
||||
- **Returns**: None
|
||||
|
||||
### `convert_time_format(time_str)`
|
||||
- **Parameters**: `time_str` (str) - Time string to convert.
|
||||
- **Returns**: Formatted time string
|
||||
|
||||
### `apply_time_format_conversion(df, columns)`
|
||||
- **Parameters**:
|
||||
- `df` (DataFrame) - DataFrame with time columns.
|
||||
- `columns` (list of str) - List of column names to format.
|
||||
- **Returns**: DataFrame with formatted time columns
|
||||
|
||||
### `main()`
|
||||
- Main function to execute the entire process: reading the Excel file, filtering data, converting time formats, creating message columns, and saving to Excel.
|
||||
|
||||
## Example
|
||||
```python
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
```
|
||||
|
||||
## Contributing
|
||||
1. Fork the repository.
|
||||
2. Create a new branch: `git checkout -b feature-branch`.
|
||||
3. Make your changes and commit: `git commit -m 'Add new feature'`.
|
||||
4. Push to the branch: `git push origin feature-branch`.
|
||||
5. Submit a pull request.
|
||||
|
||||
## License
|
||||
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for more information.
|
||||
|
||||
## Acknowledgements
|
||||
- [Pandas Documentation](https://pandas.pydata.org/pandas-docs/stable/)
|
||||
- [Openpyxl Documentation](https://openpyxl.readthedocs.io/en/stable/)
|
||||
|
||||
## Author
|
||||
- AI Assistant (Your Name or Contributors)
|
||||
|
||||
> For additional information or support, please contact `your-email@example.com`.
|
||||
@@ -1,7 +1,10 @@
|
||||
import pandas as pd
|
||||
|
||||
def read_excel_file(file_path):
|
||||
"""Read the Excel file and return a DataFrame."""
|
||||
"""
|
||||
:param file_path: The path to the Excel file to be read.
|
||||
:return: The contents of the Excel file as a DataFrame if successful, otherwise None.
|
||||
"""
|
||||
try:
|
||||
return pd.read_excel(file_path)
|
||||
except Exception as e:
|
||||
@@ -9,7 +12,10 @@ def read_excel_file(file_path):
|
||||
return None
|
||||
|
||||
def filter_studiegidsnummer(df):
|
||||
"""Filter rows where 'studiegidsnummer' contains 'GES'."""
|
||||
"""
|
||||
:param df: Input DataFrame that contains various columns including 'Studiegidsnummer'.
|
||||
:return: DataFrame filtered to include only rows where the 'Studiegidsnummer' column contains 'GES'. Returns an empty DataFrame if 'Studiegidsnummer' column is not found.
|
||||
"""
|
||||
if 'Studiegidsnummer' not in df.columns:
|
||||
print("Column 'studiegidsnummer' not found in the DataFrame.")
|
||||
print("Available columns:", df.columns)
|
||||
@@ -17,7 +23,10 @@ def filter_studiegidsnummer(df):
|
||||
return df[df['Studiegidsnummer'].str.contains('GES', na=False)].copy()
|
||||
|
||||
def filter_opmerkingen(df):
|
||||
"""Filter rows where 'Opmerkingen' does NOT contain '24-25'."""
|
||||
"""
|
||||
:param df: The input DataFrame containing various columns including 'Opmerkingen'
|
||||
:return: A filtered DataFrame excluding rows where the 'Opmerkingen' column contains the string '24-25'. If the 'Opmerkingen' column is not found, returns an empty DataFrame and prints available columns.
|
||||
"""
|
||||
if 'Opmerkingen' not in df.columns:
|
||||
print("Column 'Opmerkingen' not found in the DataFrame.")
|
||||
print("Available columns:", df.columns)
|
||||
@@ -25,7 +34,11 @@ def filter_opmerkingen(df):
|
||||
return df[~df['Opmerkingen'].str.contains('24-25', na=False)].copy()
|
||||
|
||||
def create_message_column(df):
|
||||
"""Create 'Message' and 'subject' columns with the specified format."""
|
||||
"""
|
||||
:param df: A pandas DataFrame containing examination details.
|
||||
:return: A pandas DataFrame with additional 'Message' and 'subject' columns
|
||||
for communication with teaching staff regarding examination details.
|
||||
"""
|
||||
df.loc[:, 'Message'] = df.apply(lambda row: (
|
||||
f"Beste docent,\n\n"
|
||||
f"Ik ben de examengegevens aan het controleren van {row['Omschrijving']} {row['Studiegidsnummer']}. De huidige gegevens zijn als volgt:\n\n"
|
||||
@@ -38,14 +51,23 @@ def create_message_column(df):
|
||||
return df
|
||||
|
||||
def save_to_excel(df, output_file_path):
|
||||
"""Save the DataFrame to a new Excel file."""
|
||||
"""
|
||||
:param df: The DataFrame to be saved to an Excel file.
|
||||
:type df: pandas.DataFrame
|
||||
:param output_file_path: The path where the Excel file will be saved.
|
||||
:type output_file_path: str
|
||||
:return: None
|
||||
"""
|
||||
try:
|
||||
df.to_excel(output_file_path, index=False)
|
||||
except Exception as e:
|
||||
print(f"Error saving the Excel file: {e}")
|
||||
|
||||
def convert_time_format(time_str):
|
||||
"""Convert time from 'HH:MM:SS' to 'HH:MM'."""
|
||||
"""
|
||||
:param time_str: A string representing the time to be converted.
|
||||
:return: A string representing the time in 'HH:MM' format, or the original string if conversion fails.
|
||||
"""
|
||||
try:
|
||||
return pd.to_datetime(time_str).strftime('%H:%M')
|
||||
except Exception as e:
|
||||
@@ -53,13 +75,26 @@ def convert_time_format(time_str):
|
||||
return time_str
|
||||
|
||||
def apply_time_format_conversion(df, columns):
|
||||
"""Apply time format conversion to specified columns in the DataFrame."""
|
||||
"""
|
||||
:param df: The DataFrame containing the columns to be formatted.
|
||||
:type df: pandas.DataFrame
|
||||
:param columns: A list of column names in the DataFrame to apply the time format conversion.
|
||||
:type columns: list of str
|
||||
:return: A DataFrame with the specified columns converted to the '%H:%M' format.
|
||||
:rtype: pandas.DataFrame
|
||||
"""
|
||||
for column in columns:
|
||||
df[column] = pd.to_datetime(df[column], format='%H:%M:%S', errors='coerce').dt.strftime('%H:%M')
|
||||
return df
|
||||
|
||||
# Example usage within the main function
|
||||
def main():
|
||||
"""
|
||||
Reads an Excel file, filters data based on specific criteria, converts time formats for specified columns,
|
||||
creates a message column, and saves the filtered data to a new Excel file.
|
||||
|
||||
:return: None
|
||||
"""
|
||||
file_path = 'examengegevens2425.xlsx'
|
||||
output_file_path = 'filtered_examengegevens2425.xlsx'
|
||||
|
||||
|
||||
Reference in New Issue
Block a user