Skip to content

Zip upload with or without index column #77

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
einfachMel merged 3 commits into from
Jan 17, 2022
Merged

Zip upload with or without index column #77

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
95c1963
Removed columns with no name from import of excel files
V3lop5 Jan 17, 2022
ae12057
Fixed removing of index column
V3lop5 Jan 17, 2022
e5237a2
Added dataset 2 with index column
V3lop5 Jan 17, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions ensysmod/core/file_upload.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -278,6 +278,9 @@ def process_excel_file(file: TemporaryFile, db: Session, dataset_id: int, compon

# for each column, create a time series
for column in df.columns:
if column == "Unnamed: 0":
# Skip first unnamed column, because this is might be the index
continue
request_dict = {data_key: df[column].tolist(), "region": column, "component": component_name}
create_request = map_with_dataset_id(create_model, request_dict, dataset_id)
create_or_update_time_series(crud_repo, db, create_request)
Expand Down
2 changes: 2 additions & 0 deletions examples/data/dataset-2/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Dataset 2 (Copy of Dataset 1)
... but with index column in excel files!
27 changes: 27 additions & 0 deletions examples/data/dataset-2/commodities.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
[
{
"description": "Commodity zu Strom. Die Werte werden in elektrischen GW angegeben.",
"name": "Elektrizität",
"unit": "GW"
},
{
"description": "Commodity zu Wasserstoff. Die Werte werden in GW H2 (zum Heizwert) angegeben.",
"name": "Wasserstoff",
"unit": "GW"
},
{
"description": "Commodity zu Methan. Die Werte werden in GW CH4 (zum Heizwert) angegeben.",
"name": "Methan",
"unit": "GW"
},
{
"description": "Commodity zu Bio Gas. Die Werte werden in GW Bio Gas (zum Heizwert) angegeben.",
"name": "Biogas",
"unit": "GW"
},
{
"description": "Commodity zu CO2. Die Werte werden in Mio. t CO2 pro Stunde angegeben.",
"name": "CO2",
"unit": "Mio. t CO2/h"
}
]
22 changes: 22 additions & 0 deletions examples/data/dataset-2/conversions/elektrolyseure/conversion.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"name": "Elektrolyseure",
"description": "Stellt Elektrolyseure dar",
"capacity_variable_domain": "CONTINUOUS",
"invest_per_capacity": 0.5,
"interest_rate": 0.08,
"capacity_variable": false,
"capacity_per_plant_unit": 1.0,
"opex_per_capacity": 0.0125,
"economic_lifetime": 10,
"commodity_unit": "Wasserstoff",
"conversion_factors": [
{
"conversion_factor": -1.0,
"commodity": "Elektrizität"
},
{
"conversion_factor": 0.7,
"commodity": "Wasserstoff"
}
]
}
22 changes: 22 additions & 0 deletions examples/data/dataset-2/conversions/gud-turbine-(biogas)/conversion.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"name": "GuD-Turbine (Biogas)",
"description": "Stellt Gas-und Dampfturbinen für Biogas dar",
"capacity_variable_domain": "CONTINUOUS",
"invest_per_capacity": 0.7,
"interest_rate": 0.08,
"capacity_variable": false,
"capacity_per_plant_unit": 1.0,
"opex_per_capacity": 0.021,
"economic_lifetime": 33,
"commodity_unit": "Biogas",
"conversion_factors": [
{
"conversion_factor": 1.0,
"commodity": "Elektrizität"
},
{
"conversion_factor": -1.5873,
"commodity": "Biogas"
}
]
}
22 changes: 22 additions & 0 deletions examples/data/dataset-2/conversions/gud-turbine-(methan)/conversion.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"name": "GuD-Turbine (Methan)",
"description": "Stellt Gas-und Dampfturbinen für Methan dar",
"capacity_variable_domain": "CONTINUOUS",
"invest_per_capacity": 0.0,
"interest_rate": 0.08,
"capacity_variable": false,
"capacity_per_plant_unit": 1.0,
"opex_per_capacity": 0.0,
"economic_lifetime": 10,
"commodity_unit": "Methan",
"conversion_factors": [
{
"conversion_factor": 0.6,
"commodity": "Elektrizität"
},
{
"conversion_factor": 1.0,
"commodity": "Methan"
}
]
}
22 changes: 22 additions & 0 deletions examples/data/dataset-2/conversions/gud-turbine-(wasserstoff)/conversion.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"name": "GuD-Turbine (Wasserstoff)",
"description": "Stellt Gas-und Dampfturbinen für Wasserstoff dar",
"capacity_variable_domain": "CONTINUOUS",
"invest_per_capacity": 0.7,
"interest_rate": 0.08,
"capacity_variable": false,
"capacity_per_plant_unit": 1.0,
"opex_per_capacity": 0.021,
"economic_lifetime": 33,
"commodity_unit": "Wasserstoff",
"conversion_factors": [
{
"conversion_factor": 1.0,
"commodity": "Elektrizität"
},
{
"conversion_factor": -1.5873,
"commodity": "Wasserstoff"
}
]
}
22 changes: 22 additions & 0 deletions examples/data/dataset-2/conversions/rsoec/conversion.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"name": "rSOEC",
"description": "Stellt rSOEC dar",
"capacity_variable_domain": "CONTINUOUS",
"invest_per_capacity": 0.75,
"interest_rate": 0.08,
"capacity_variable": false,
"capacity_per_plant_unit": 1.0,
"opex_per_capacity": 0.015,
"economic_lifetime": 10,
"commodity_unit": "Wasserstoff",
"conversion_factors": [
{
"conversion_factor": -1.0,
"commodity": "Elektrizität"
},
{
"conversion_factor": 0.6,
"commodity": "Wasserstoff"
}
]
}
22 changes: 22 additions & 0 deletions examples/data/dataset-2/conversions/rsofc/conversion.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"name": "rSOFC",
"description": "Stellt rSOFC dar",
"capacity_variable_domain": "CONTINUOUS",
"invest_per_capacity": 0.75,
"interest_rate": 0.08,
"capacity_variable": false,
"capacity_per_plant_unit": 1.0,
"opex_per_capacity": 0.015,
"economic_lifetime": 10,
"commodity_unit": "Wasserstoff",
"conversion_factors": [
{
"conversion_factor": 1.0,
"commodity": "Elektrizität"
},
{
"conversion_factor": -1.66667,
"commodity": "Wasserstoff"
}
]
}
26 changes: 26 additions & 0 deletions examples/data/dataset-2/regions.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
[
{
"name": "Cluster 0"
},
{
"name": "Cluster 1"
},
{
"name": "Cluster 2"
},
{
"name": "Cluster 3"
},
{
"name": "Cluster 4"
},
{
"name": "Cluster 5"
},
{
"name": "Cluster 6"
},
{
"name": "Cluster 7"
}
]
Binary file added examples/data/dataset-2/sinks/co2-verbrauch/operationRateFix.xlsx
View file
Open in desktop
Binary file not shown.
12 changes: 12 additions & 0 deletions examples/data/dataset-2/sinks/co2-verbrauch/sink.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"name": "CO2-Verbrauch",
"description": "Stellt den CO2-Verbrauch dar",
"capacity_variable_domain": "CONTINUOUS",
"invest_per_capacity": 0.0,
"interest_rate": 0.08,
"capacity_variable": false,
"capacity_per_plant_unit": 1.0,
"opex_per_capacity": 0.0,
"economic_lifetime": 10,
"commodity": "CO2"
}
Binary file added examples/data/dataset-2/sinks/elektrizität-verbrauch/operationRateFix.xlsx
View file
Open in desktop
Binary file not shown.
12 changes: 12 additions & 0 deletions examples/data/dataset-2/sinks/elektrizität-verbrauch/sink.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"name": "Elektrizität-Verbrauch",
"description": "Stellt den Elektrizität-Verbrauch dar",
"capacity_variable_domain": "CONTINUOUS",
"invest_per_capacity": 0.0,
"interest_rate": 0.08,
"capacity_variable": false,
"capacity_per_plant_unit": 1.0,
"opex_per_capacity": 0.0,
"economic_lifetime": 10,
"commodity": "Elektrizität"
}
Binary file added examples/data/dataset-2/sinks/wasserstoff-verbrauch/operationRateFix.xlsx
View file
Open in desktop
Binary file not shown.
12 changes: 12 additions & 0 deletions examples/data/dataset-2/sinks/wasserstoff-verbrauch/sink.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"name": "Wasserstoff-Verbrauch",
"description": "Stellt den Wasserstoff-Verbrauch dar",
"capacity_variable_domain": "CONTINUOUS",
"invest_per_capacity": 0.0,
"interest_rate": 0.08,
"capacity_variable": false,
"capacity_per_plant_unit": 1.0,
"opex_per_capacity": 0.0,
"economic_lifetime": 10,
"commodity": "Wasserstoff"
}
Binary file added examples/data/dataset-2/sources/biogas-importe/operationRateMax.xlsx
View file
Open in desktop
Binary file not shown.
13 changes: 13 additions & 0 deletions examples/data/dataset-2/sources/biogas-importe/source.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"name": "Biogas Importe",
"description": "Stellt die Biogas-Importe dar",
"capacity_variable_domain": "CONTINUOUS",
"invest_per_capacity": 0.0,
"interest_rate": 0.08,
"capacity_variable": false,
"capacity_per_plant_unit": 1.0,
"opex_per_capacity": 0.0,
"economic_lifetime": 10,
"commodity_cost": 5.409e-05,
"commodity": "Biogas"
}
13 changes: 13 additions & 0 deletions examples/data/dataset-2/sources/natural-gas-importe/source.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
{
"name": "Natural Gas Importe",
"description": "Stellt die Natural Gas-Importe dar",
"capacity_variable_domain": "CONTINUOUS",
"invest_per_capacity": 0.0,
"interest_rate": 0.08,
"capacity_variable": false,
"capacity_per_plant_unit": 1.0,
"opex_per_capacity": 0.0,
"economic_lifetime": 10,
"commodity_cost": 3.31e-05,
"commodity": "Methan"
}
Binary file added examples/data/dataset-2/sources/pv/capacityMax.xlsx
View file
Open in desktop
Binary file not shown.
Binary file added examples/data/dataset-2/sources/pv/operationRateMax.xlsx
View file
Open in desktop
Binary file not shown.
12 changes: 12 additions & 0 deletions examples/data/dataset-2/sources/pv/source.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"name": "PV",
"description": "Stellt die PV-Anlagen dar",
"capacity_variable_domain": "CONTINUOUS",
"invest_per_capacity": 0.65,
"interest_rate": 0.08,
"capacity_variable": false,
"capacity_per_plant_unit": 1.0,
"opex_per_capacity": 0.013,
"economic_lifetime": 25,
"commodity": "Elektrizität"
}
Binary file added examples/data/dataset-2/sources/run-of-river/capacityFix.xlsx
View file
Open in desktop
Binary file not shown.
Binary file added examples/data/dataset-2/sources/run-of-river/operationRateFix.xlsx
View file
Open in desktop
Binary file not shown.
12 changes: 12 additions & 0 deletions examples/data/dataset-2/sources/run-of-river/source.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"name": "Run-of-river",
"description": "Stellt die Laufwasser-Kraftwerke dar",
"capacity_variable_domain": "CONTINUOUS",
"invest_per_capacity": 0.0,
"interest_rate": 0.08,
"capacity_variable": false,
"capacity_per_plant_unit": 1.0,
"opex_per_capacity": 0.208,
"economic_lifetime": 10,
"commodity": "Elektrizität"
}
Binary file added examples/data/dataset-2/sources/wind-offshore/capacityMax.xlsx
View file
Open in desktop
Binary file not shown.
Binary file added examples/data/dataset-2/sources/wind-offshore/operationRateMax.xlsx
View file
Open in desktop
Binary file not shown.
12 changes: 12 additions & 0 deletions examples/data/dataset-2/sources/wind-offshore/source.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"name": "Wind Offshore",
"description": "Stellt die Wind Offshore-Anlagen dar",
"capacity_variable_domain": "CONTINUOUS",
"invest_per_capacity": 2.3,
"interest_rate": 0.08,
"capacity_variable": false,
"capacity_per_plant_unit": 1.0,
"opex_per_capacity": 0.046,
"economic_lifetime": 20,
"commodity": "Elektrizität"
}
Binary file added examples/data/dataset-2/sources/wind-onshore/capacityMax.xlsx
View file
Open in desktop
Binary file not shown.
Binary file added examples/data/dataset-2/sources/wind-onshore/operationRateMax.xlsx
View file
Open in desktop
Binary file not shown.
12 changes: 12 additions & 0 deletions examples/data/dataset-2/sources/wind-onshore/source.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"name": "Wind Onshore",
"description": "Stellt die Wind Onshore-Anlagen dar",
"capacity_variable_domain": "CONTINUOUS",
"invest_per_capacity": 1.1,
"interest_rate": 0.08,
"capacity_variable": false,
"capacity_per_plant_unit": 1.0,
"opex_per_capacity": 0.022,
"economic_lifetime": 20,
"commodity": "Elektrizität"
}
18 changes: 18 additions & 0 deletions examples/data/dataset-2/storages/li-ion-batterien/storage.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"name": "Li-ion Batterien",
"description": "Stellt Batterien dar",
"capacity_variable_domain": "CONTINUOUS",
"invest_per_capacity": 0.151,
"interest_rate": 0.08,
"capacity_variable": false,
"capacity_per_plant_unit": 1.0,
"opex_per_capacity": 0.002,
"economic_lifetime": 22,
"charge_efficiency": 0.95,
"self_discharge": 4.23036e-05,
"charge_rate": 1.0,
"discharge_efficiency": 0.95,
"cyclic_lifetime": 10000,
"discharge_rate": 1.0,
"commodity": "Elektrizität"
}
Binary file added examples/data/dataset-2/storages/pumpspeicher-kraftwerk/capacityFix.xlsx
View file
Open in desktop
Binary file not shown.
17 changes: 17 additions & 0 deletions examples/data/dataset-2/storages/pumpspeicher-kraftwerk/storage.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"name": "Pumpspeicher-Kraftwerk",
"description": "Stellt Pumpspeicher-Kraftwerk dar",
"capacity_variable_domain": "CONTINUOUS",
"invest_per_capacity": 0.0,
"interest_rate": 0.08,
"capacity_variable": false,
"capacity_per_plant_unit": 1.0,
"opex_per_capacity": 0.000153,
"economic_lifetime": 10,
"charge_efficiency": 0.88,
"self_discharge": 5.2181e-06,
"charge_rate": 0.16,
"discharge_efficiency": 0.88,
"discharge_rate": 0.12,
"commodity": "Elektrizität"
}
Binary file added examples/data/dataset-2/storages/salzkavernen-(bio-gas)/capacityMax.xlsx
View file
Open in desktop
Binary file not shown.
Loading