Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Requests to ComEd fail with too large of a date range #32

Open
bjackson opened this issue Aug 10, 2023 · 6 comments
Open

Requests to ComEd fail with too large of a date range #32

bjackson opened this issue Aug 10, 2023 · 6 comments
Labels
bug Something isn't working

Comments

@bjackson
Copy link

In reference to home-assistant/core#97814.

The HA integration requests day aggregate information from ComEd from 2022-06-13T00:00:00-05:00 to 2023-06-11T00:00:00-05:00. This fails with the following response:

{
	"error": {
		"httpStatus": 500,
		"serviceErrorCode": "UPSTREAM_ERROR",
		"details": "Could not get rated costs and usages for utility account UUID: XXX"
	}
}

Here's what I ran in the terminal:

➜ python src/demo.py --utility comed --username "[email protected]" --password 'XXX' --start_date 2022-06-13T00:00:00-05:00 --end_date 2023-06-11T00:00:00-05:00

Current bill forecast: Forecast(account=Account(customer=Customer(uuid='9e967f23-9e95-11eb-bf8b-0200170058ac'), uuid='a3ba62b5-9e95-11eb-bf8b-0200170058ac', utility_account_id='XXX', meter_type=<MeterType.ELEC: 'ELEC'>, read_resolution=None), start_date=datetime.date(2023, 8, 1), end_date=datetime.date(2023, 8, 30), current_date=datetime.date(2023, 8, 10), unit_of_measure=<UnitOfMeasure.KWH: 'KWH'>, usage_to_date=309.0, cost_to_date=0.0, forecasted_usage=1337.0, forecasted_cost=0.0, typical_usage=1099.0, typical_cost=0.0)

Getting historical data: account= Account(customer=Customer(uuid='9e967f23-9e95-11eb-bf8b-0200170058ac'), uuid='a3ba62b5-9e95-11eb-bf8b-0200170058ac', utility_account_id='XXX', meter_type=<MeterType.ELEC: 'ELEC'>, read_resolution=<ReadResolution.HALF_HOUR: 'HALF_HOUR'>) aggregate_type= day start_date= 2022-06-13 00:00:00-05:00 end_date= 2023-06-11 00:00:00-05:00
Traceback (most recent call last):
  File "/Users/brett/pcode/opower/src/demo.py", line 151, in <module>
    asyncio.run(_main())
  File "/Users/brett/.pyenv/versions/3.10.1/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Users/brett/.pyenv/versions/3.10.1/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete
    return future.result()
  File "/Users/brett/pcode/opower/src/demo.py", line 121, in _main
    cost_data = await opower.async_get_cost_reads(
  File "/Users/brett/pcode/opower/src/opower/opower.py", line 327, in async_get_cost_reads
    reads = await self._async_get_dated_data(
  File "/Users/brett/pcode/opower/src/opower/opower.py", line 431, in _async_get_dated_data
    reads = await self._async_fetch(
  File "/Users/brett/pcode/opower/src/opower/opower.py", line 475, in _async_fetch
    raise err
  File "/Users/brett/pcode/opower/src/opower/opower.py", line 463, in _async_fetch
    async with self.session.get(
  File "/Users/brett/.virtualenvs/opower-quohth9L/lib/python3.10/site-packages/aiohttp/client.py", line 1141, in __aenter__
    self._resp = await self._coro
  File "/Users/brett/.virtualenvs/opower-quohth9L/lib/python3.10/site-packages/aiohttp/client.py", line 643, in _request
    resp.raise_for_status()
  File "/Users/brett/.virtualenvs/opower-quohth9L/lib/python3.10/site-packages/aiohttp/client_reqrep.py", line 1005, in raise_for_status
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 500, message='Internal Server Error', url=URL('https://cec.opower.com/ei/edge/apis/DataBrowser-v1/cws/cost/utilityAccount/a3ba62b5-9e95-11eb-bf8b-0200170058ac?aggregateType=day&startDate=2022-06-14T00:00:00-05:00&endDate=2023-06-12T00:00:00-05:00')

This data is available on ComEd's website.

However, if I change the time frame with opower's demo.py, keeping the start date the same, but decreasing the end date, the request succeeds. I think that opower may be limiting the number of days for which data can be requested, and possibly may be configured per utility.

➜ python src/demo.py --utility comed --username "[email protected]" --password 'XXX' --start_date 2022-06-13T00:00:00-05:00 --end_date 2022-10-11T00:00:00-05:00

Current bill forecast: Forecast(account=Account(customer=Customer(uuid='9e967f23-9e95-11eb-bf8b-0200170058ac'), uuid='a3ba62b5-9e95-11eb-bf8b-0200170058ac', utility_account_id='XXX', meter_type=<MeterType.ELEC: 'ELEC'>, read_resolution=None), start_date=datetime.date(2023, 8, 1), end_date=datetime.date(2023, 8, 30), current_date=datetime.date(2023, 8, 10), unit_of_measure=<UnitOfMeasure.KWH: 'KWH'>, usage_to_date=309.0, cost_to_date=0.0, forecasted_usage=1337.0, forecasted_cost=0.0, typical_usage=1099.0, typical_cost=0.0)

Getting historical data: account= Account(customer=Customer(uuid='9e967f23-9e95-11eb-bf8b-0200170058ac'), uuid='a3ba62b5-9e95-11eb-bf8b-0200170058ac', utility_account_id='XXX', meter_type=<MeterType.ELEC: 'ELEC'>, read_resolution=<ReadResolution.HALF_HOUR: 'HALF_HOUR'>) aggregate_type= day start_date= 2022-06-13 00:00:00-05:00 end_date= 2022-10-11 00:00:00-05:00
start_time	end_time	consumption	provided_cost	start_minus_prev_end	end_minus_prev_end
2022-06-13 00:00:00-05:00	2022-06-14 00:00:00-05:00	28.9075	4.626645375	None	None
2022-06-14 00:00:00-05:00	2022-06-15 00:00:00-05:00	41.215	6.59646075	0:00:00	1 day, 0:00:00
2022-06-15 00:00:00-05:00	2022-06-16 00:00:00-05:00	59.275	9.48696375	0:00:00	1 day, 0:00:00
2022-06-16 00:00:00-05:00	2022-06-17 00:00:00-05:00	51.265	8.20496325	0:00:00	1 day, 0:00:00
2022-06-17 00:00:00-05:00	2022-06-18 00:00:00-05:00	36.6475	5.865432375	0:00:00	1 day, 0:00:00
2022-06-18 00:00:00-05:00	2022-06-19 00:00:00-05:00	19.6875	3.150984375	0:00:00	1 day, 0:00:00
2022-06-19 00:00:00-05:00	2022-06-20 00:00:00-05:00	20.095	3.21620475	0:00:00	1 day, 0:00:00
...

My account has been active since sometime in April/May 2021 and had a smart meter before activation.

If you'd like access to my utility account, let me know an email and I can share credentials or a HAR file of requests on ComEd's usage page with you.
@bjackson bjackson changed the title Requests to ComEd fail before a certain date Requests to ComEd fail with too large of a date range Aug 10, 2023
@tronikos
Copy link
Owner

Can you try lowering max_request_days at https://github.com/tronikos/opower/blob/main/src/opower/opower.py#L416 ?
Try also with --aggregate_type hour to see if you need to lower the value two lines below.

@bjackson
Copy link
Author

bjackson commented Aug 10, 2023 •
edited
Loading

It appears the max max_request_days that works is dependent on the times I set.

For example, I run with --start_date 2022-10-13T00:00:00-05:00 --end_date 2023-03-12T00:00:00-06:00 and max_request_days of 140, then it fails. It starts working when max_request <= 126. The URL that fails when max_request_days = 127 is https://cec.opower.com/ei/edge/apis/DataBrowser-v1/cws/cost/utilityAccount/a3ba62b5-9e95-11eb-bf8b-0200170058ac?aggregateType=day&startDate=2022-11-06T00:00:00-05:00&endDate=2023-03-13T00:00:00-05:00'

But if I run it with --start_date 2022-10-13T00:00:00-05:00 --end_date 2023-*05*-12T00:00:00-06:00 and keep it at 140 days, it succeeds.

Running --start_date 2022-10-13T00:00:00-05:00 --end_date 2023-03-12T00:00:00-06:00 --aggregate_type hour works fine with the default value.

Here are the debug logs for day aggregate requests:

`max_req = 120` 
/Users/brett/.virtualenvs/opower-quohth9L/bin/python /Users/brett/pcode/opower/src/demo.py --utility comed --username [email protected] --password XXX --start_date 2022-10-13T00:00:00-05:00 --end_date 2023-03-12T00:00:00-06:00 --aggregate_type day --verbose 
DEBUG:/Users/brett/pcode/opower/src/opower/opower.py:Fetching: https://cec.opower.com/ei/edge/apis/multi-account-v1/cws/cec/customers?offset=0&batchSize=100&addressFilter=
DEBUG:/Users/brett/pcode/opower/src/opower/opower.py:Fetching: https://cec.opower.com/ei/edge/apis/bill-forecast-cws-v1/cws/cec/customers/9e967f23-9e95-11eb-bf8b-0200170058ac/combined-forecast
DEBUG:/Users/brett/pcode/opower/src/opower/opower.py:Fetching: https://cec.opower.com/ei/edge/apis/DataBrowser-v1/cws/cost/utilityAccount/a3ba62b5-9e95-11eb-bf8b-0200170058ac?aggregateType=day&startDate=2022-11-13T00%3A00%3A00-06%3A00&endDate=2023-03-13T00%3A00%3A00-05%3A00

Current bill forecast: Forecast(account=Account(customer=Customer(uuid='9e967f23-9e95-11eb-bf8b-0200170058ac'), uuid='a3ba62b5-9e95-11eb-bf8b-0200170058ac', utility_account_id='0855043257', meter_type=<MeterType.ELEC: 'ELEC'>, read_resolution=None), start_date=datetime.date(2023, 8, 1), end_date=datetime.date(2023, 8, 30), current_date=datetime.date(2023, 8, 10), unit_of_measure=<UnitOfMeasure.KWH: 'KWH'>, usage_to_date=309.0, cost_to_date=0.0, forecasted_usage=1337.0, forecasted_cost=0.0, typical_usage=1099.0, typical_cost=0.0)


Getting historical data: account= Account(customer=Customer(uuid='9e967f23-9e95-11eb-bf8b-0200170058ac'), uuid='a3ba62b5-9e95-11eb-bf8b-0200170058ac', utility_account_id='0855043257', meter_type=<MeterType.ELEC: 'ELEC'>, read_resolution=<ReadResolution.HALF_HOUR: 'HALF_HOUR'>) aggregate_type= day start_date= 2022-10-13 00:00:00-05:00 end_date= 2023-03-12 00:00:00-06:00

DEBUG:/Users/brett/pcode/opower/src/opower/opower.py:Fetching: https://cec.opower.com/ei/edge/apis/DataBrowser-v1/cws/cost/utilityAccount/a3ba62b5-9e95-11eb-bf8b-0200170058ac?aggregateType=day&startDate=2022-10-13T00%3A00%3A00-05%3A00&endDate=2022-11-12T00%3A00%3A00-06%3A00

start_time	end_time	consumption	provided_cost	start_minus_prev_end	end_minus_prev_end

2022-10-13 00:00:00-05:00	2022-10-14 00:00:00-05:00	12.9	1.899138	None	None

2022-10-14 00:00:00-05:00	2022-10-15 00:00:00-05:00	12.9325	1.90392265	0:00:00	1 day, 0:00:00

2022-10-15 00:00:00-05:00	2022-10-16 00:00:00-05:00	9.9675	1.46741535	0:00:00	1 day, 0:00:00

2022-10-16 00:00:00-05:00	2022-10-17 00:00:00-05:00	15.535	2.2870627	0:00:00	1 day, 0:00:00

2022-10-17 00:00:00-05:00	2022-10-18 00:00:00-05:00	12.9175	1.90171435	0:00:00	1 day, 0:00:00

2022-10-18 00:00:00-05:00	2022-10-19 00:00:00-05:00	17.7925	2.61941185	0:00:00	1 day, 0:00:00

2022-10-19 00:00:00-05:00	2022-10-20 00:00:00-05:00	22.71	3.3433662	0:00:00	1 day, 0:00:00

2022-10-20 00:00:00-05:00	2022-10-21 00:00:00-05:00	19.55	2.878151	0:00:00	1 day, 0:00:00

2022-10-21 00:00:00-05:00	2022-10-22 00:00:00-05:00	33.99	5.0040078	0:00:00	1 day, 0:00:00

2022-10-22 00:00:00-05:00	2022-10-23 00:00:00-05:00	38.21	5.6252762	0:00:00	1 day, 0:00:00

2022-10-23 00:00:00-05:00	2022-10-24 00:00:00-05:00	42.1425	6.20421885	0:00:00	1 day, 0:00:00

2022-10-24 00:00:00-05:00	2022-10-25 00:00:00-05:00	18.815	2.7699443	0:00:00	1 day, 0:00:00

2022-10-25 00:00:00-05:00	2022-10-26 00:00:00-05:00	21.21	3.1225362	0:00:00	1 day, 0:00:00

2022-10-26 00:00:00-05:00	2022-10-27 00:00:00-05:00	16.5975	2.44348395	0:00:00	1 day, 0:00:00

2022-10-27 00:00:00-05:00	2022-10-28 00:00:00-05:00	11.545	1.6996549	0:00:00	1 day, 0:00:00

2022-10-28 00:00:00-05:00	2022-10-29 00:00:00-05:00	11.9325	1.75670265	0:00:00	1 day, 0:00:00

2022-10-29 00:00:00-05:00	2022-10-30 00:00:00-05:00	11.5725	1.70370345	0:00:00	1 day, 0:00:00

2022-10-30 00:00:00-05:00	2022-10-31 00:00:00-05:00	12.6625	1.86417325	0:00:00	1 day, 0:00:00

2022-10-31 00:00:00-05:00	2022-11-01 00:00:00-05:00	15.7175	2.31393035	0:00:00	1 day, 0:00:00

2022-11-01 00:00:00-05:00	2022-11-02 00:00:00-05:00	18.41	2.7103202	0:00:00	1 day, 0:00:00

2022-11-02 00:00:00-05:00	2022-11-03 00:00:00-05:00	20.6025	3.03310005	0:00:00	1 day, 0:00:00

2022-11-03 00:00:00-05:00	2022-11-04 00:00:00-05:00	8.7375	1.28633475	0:00:00	1 day, 0:00:00

2022-11-04 00:00:00-05:00	2022-11-05 00:00:00-05:00	5.1475	0.75781495	0:00:00	1 day, 0:00:00

2022-11-05 00:00:00-05:00	2022-11-06 00:00:00-05:00	5.115	0.7530303	0:00:00	1 day, 0:00:00

2022-11-06 00:00:00-05:00	2022-11-07 00:00:00-06:00	5.92	0.8715424	0:00:00	1 day, 1:00:00

2022-11-07 00:00:00-06:00	2022-11-08 00:00:00-06:00	10.69	1.5737818	0:00:00	1 day, 0:00:00

2022-11-08 00:00:00-06:00	2022-11-09 00:00:00-06:00	11.07	1.6297254	0:00:00	1 day, 0:00:00

2022-11-09 00:00:00-06:00	2022-11-10 00:00:00-06:00	10.63	1.5649486	0:00:00	1 day, 0:00:00

2022-11-10 00:00:00-06:00	2022-11-11 00:00:00-06:00	17.445	2.5682529	0:00:00	1 day, 0:00:00

2022-11-11 00:00:00-06:00	2022-11-12 00:00:00-06:00	8.2025	1.20757205	0:00:00	1 day, 0:00:00

2022-11-12 00:00:00-06:00	2022-11-13 00:00:00-06:00	4.7275	0.69598255	0:00:00	1 day, 0:00:00

2022-11-13 00:00:00-06:00	2022-11-14 00:00:00-06:00	8.2725	1.21787745	0:00:00	1 day, 0:00:00

2022-11-14 00:00:00-06:00	2022-11-15 00:00:00-06:00	12.04	1.7725288	0:00:00	1 day, 0:00:00

2022-11-15 00:00:00-06:00	2022-11-16 00:00:00-06:00	10.28	1.5134216	0:00:00	1 day, 0:00:00

2022-11-16 00:00:00-06:00	2022-11-17 00:00:00-06:00	10.3075	1.51747015	0:00:00	1 day, 0:00:00

2022-11-17 00:00:00-06:00	2022-11-18 00:00:00-06:00	12.945	1.9057629	0:00:00	1 day, 0:00:00

2022-11-18 00:00:00-06:00	2022-11-19 00:00:00-06:00	14.4625	2.12916925	0:00:00	1 day, 0:00:00

2022-11-19 00:00:00-06:00	2022-11-20 00:00:00-06:00	15.3175	2.25504235	0:00:00	1 day, 0:00:00

2022-11-20 00:00:00-06:00	2022-11-21 00:00:00-06:00	14.2175	2.09310035	0:00:00	1 day, 0:00:00

2022-11-21 00:00:00-06:00	2022-11-22 00:00:00-06:00	13.4975	1.98710195	0:00:00	1 day, 0:00:00

2022-11-22 00:00:00-06:00	2022-11-23 00:00:00-06:00	8.1225	1.19579445	0:00:00	1 day, 0:00:00

2022-11-23 00:00:00-06:00	2022-11-24 00:00:00-06:00	7.1575	1.05372715	0:00:00	1 day, 0:00:00

2022-11-24 00:00:00-06:00	2022-11-25 00:00:00-06:00	7.1225	1.04857445	0:00:00	1 day, 0:00:00

2022-11-25 00:00:00-06:00	2022-11-26 00:00:00-06:00	7.125	1.0489425	0:00:00	1 day, 0:00:00

2022-11-26 00:00:00-06:00	2022-11-27 00:00:00-06:00	6.885	1.0136097	0:00:00	1 day, 0:00:00

2022-11-27 00:00:00-06:00	2022-11-28 00:00:00-06:00	9.865	1.4523253	0:00:00	1 day, 0:00:00

2022-11-28 00:00:00-06:00	2022-11-29 00:00:00-06:00	11.1425	1.64039885	0:00:00	1 day, 0:00:00

2022-11-29 00:00:00-06:00	2022-11-30 00:00:00-06:00	12.615	1.8571803	0:00:00	1 day, 0:00:00

2022-11-30 00:00:00-06:00	2022-12-01 00:00:00-06:00	14.2875	2.10340575	0:00:00	1 day, 0:00:00

2022-12-01 00:00:00-06:00	2022-12-02 00:00:00-06:00	12.81	1.9126611	0:00:00	1 day, 0:00:00

2022-12-02 00:00:00-06:00	2022-12-03 00:00:00-06:00	23.1825	3.461379075	0:00:00	1 day, 0:00:00

2022-12-03 00:00:00-06:00	2022-12-04 00:00:00-06:00	15.8025	2.359471275	0:00:00	1 day, 0:00:00

2022-12-04 00:00:00-06:00	2022-12-05 00:00:00-06:00	14.065	2.10004515	0:00:00	1 day, 0:00:00

2022-12-05 00:00:00-06:00	2022-12-06 00:00:00-06:00	15.295	2.28369645	0:00:00	1 day, 0:00:00

2022-12-06 00:00:00-06:00	2022-12-07 00:00:00-06:00	12.665	1.89101115	0:00:00	1 day, 0:00:00

2022-12-07 00:00:00-06:00	2022-12-08 00:00:00-06:00	15.57	2.3247567	0:00:00	1 day, 0:00:00

2022-12-08 00:00:00-06:00	2022-12-09 00:00:00-06:00	15.5525	2.322143775	0:00:00	1 day, 0:00:00

2022-12-09 00:00:00-06:00	2022-12-10 00:00:00-06:00	25.0375	3.738349125	0:00:00	1 day, 0:00:00

2022-12-10 00:00:00-06:00	2022-12-11 00:00:00-06:00	21.65	3.2325615	0:00:00	1 day, 0:00:00

2022-12-11 00:00:00-06:00	2022-12-12 00:00:00-06:00	34.99	5.2243569	0:00:00	1 day, 0:00:00

2022-12-12 00:00:00-06:00	2022-12-13 00:00:00-06:00	18.7275	2.796203025	0:00:00	1 day, 0:00:00

2022-12-13 00:00:00-06:00	2022-12-14 00:00:00-06:00	17.775	2.65398525	0:00:00	1 day, 0:00:00

2022-12-14 00:00:00-06:00	2022-12-15 00:00:00-06:00	27.7075	4.137006825	0:00:00	1 day, 0:00:00

2022-12-15 00:00:00-06:00	2022-12-16 00:00:00-06:00	19.1275	2.855927025	0:00:00	1 day, 0:00:00

2022-12-16 00:00:00-06:00	2022-12-17 00:00:00-06:00	22.4825	3.356862075	0:00:00	1 day, 0:00:00

2022-12-17 00:00:00-06:00	2022-12-18 00:00:00-06:00	25.695	3.83652045	0:00:00	1 day, 0:00:00

2022-12-18 00:00:00-06:00	2022-12-19 00:00:00-06:00	44.885	6.70177935	0:00:00	1 day, 0:00:00

2022-12-19 00:00:00-06:00	2022-12-20 00:00:00-06:00	34.26	5.1153606	0:00:00	1 day, 0:00:00

2022-12-20 00:00:00-06:00	2022-12-21 00:00:00-06:00	29.68	4.4315208	0:00:00	1 day, 0:00:00

2022-12-21 00:00:00-06:00	2022-12-22 00:00:00-06:00	12.1925	1.820462175	0:00:00	1 day, 0:00:00

2022-12-22 00:00:00-06:00	2022-12-23 00:00:00-06:00	8.6	1.284066	0:00:00	1 day, 0:00:00

2022-12-23 00:00:00-06:00	2022-12-24 00:00:00-06:00	10.6275	1.586792025	0:00:00	1 day, 0:00:00

2022-12-24 00:00:00-06:00	2022-12-25 00:00:00-06:00	13.125	1.95969375	0:00:00	1 day, 0:00:00

2022-12-25 00:00:00-06:00	2022-12-26 00:00:00-06:00	9.6775	1.444947525	0:00:00	1 day, 0:00:00

2022-12-26 00:00:00-06:00	2022-12-27 00:00:00-06:00	9.2525	1.381490775	0:00:00	1 day, 0:00:00

2022-12-27 00:00:00-06:00	2022-12-28 00:00:00-06:00	9.3825	1.400901075	0:00:00	1 day, 0:00:00

2022-12-28 00:00:00-06:00	2022-12-29 00:00:00-06:00	12.45	1.8589095	0:00:00	1 day, 0:00:00

2022-12-29 00:00:00-06:00	2022-12-30 00:00:00-06:00	44.51	6.6457881	0:00:00	1 day, 0:00:00

2022-12-30 00:00:00-06:00	2022-12-31 00:00:00-06:00	19.4275	2.900720025	0:00:00	1 day, 0:00:00

2022-12-31 00:00:00-06:00	2023-01-01 00:00:00-06:00	8.6325	1.288918575	0:00:00	1 day, 0:00:00

2023-01-01 00:00:00-06:00	2023-01-02 00:00:00-06:00	8.0925	1.208291175	0:00:00	1 day, 0:00:00

2023-01-02 00:00:00-06:00	2023-01-03 00:00:00-06:00	24.2625	3.622633875	0:00:00	1 day, 0:00:00

2023-01-03 00:00:00-06:00	2023-01-04 00:00:00-06:00	43.3025	6.465496275	0:00:00	1 day, 0:00:00

2023-01-04 00:00:00-06:00	2023-01-05 00:00:00-06:00	38.46	3.9192405	0:00:00	1 day, 0:00:00

2023-01-05 00:00:00-06:00	2023-01-06 00:00:00-06:00	23.795	2.2676012	0:00:00	1 day, 0:00:00

2023-01-06 00:00:00-06:00	2023-01-07 00:00:00-06:00	26.275	2.8827161	0:00:00	1 day, 0:00:00

2023-01-07 00:00:00-06:00	2023-01-08 00:00:00-06:00	24.9875	2.43896185	0:00:00	1 day, 0:00:00

2023-01-08 00:00:00-06:00	2023-01-09 00:00:00-06:00	17.8925	1.705321225	0:00:00	1 day, 0:00:00

2023-01-09 00:00:00-06:00	2023-01-10 00:00:00-06:00	20.3	1.8965251	0:00:00	1 day, 0:00:00

2023-01-10 00:00:00-06:00	2023-01-11 00:00:00-06:00	24.695	3.2049756	0:00:00	1 day, 0:00:00

2023-01-11 00:00:00-06:00	2023-01-12 00:00:00-06:00	17.77	1.725693375	0:00:00	1 day, 0:00:00

2023-01-12 00:00:00-06:00	2023-01-13 00:00:00-06:00	23.73	2.122299975	0:00:00	1 day, 0:00:00

2023-01-13 00:00:00-06:00	2023-01-14 00:00:00-06:00	18.8025	2.04648485	0:00:00	1 day, 0:00:00

2023-01-14 00:00:00-06:00	2023-01-15 00:00:00-06:00	22.1425	2.5160867	0:00:00	1 day, 0:00:00

2023-01-15 00:00:00-06:00	2023-01-16 00:00:00-06:00	15.25	1.286492075	0:00:00	1 day, 0:00:00

2023-01-16 00:00:00-06:00	2023-01-17 00:00:00-06:00	32.0	2.8796108	0:00:00	1 day, 0:00:00

2023-01-17 00:00:00-06:00	2023-01-18 00:00:00-06:00	41.02	3.688363075	0:00:00	1 day, 0:00:00

2023-01-18 00:00:00-06:00	2023-01-19 00:00:00-06:00	34.15	3.247907975	0:00:00	1 day, 0:00:00

2023-01-19 00:00:00-06:00	2023-01-20 00:00:00-06:00	27.4875	2.4340504	0:00:00	1 day, 0:00:00

2023-01-20 00:00:00-06:00	2023-01-21 00:00:00-06:00	39.535	3.677085675	0:00:00	1 day, 0:00:00

2023-01-21 00:00:00-06:00	2023-01-22 00:00:00-06:00	33.88	3.33415245	0:00:00	1 day, 0:00:00

2023-01-22 00:00:00-06:00	2023-01-23 00:00:00-06:00	31.07	2.961562125	0:00:00	1 day, 0:00:00

2023-01-23 00:00:00-06:00	2023-01-24 00:00:00-06:00	28.7175	2.60095755	0:00:00	1 day, 0:00:00

2023-01-24 00:00:00-06:00	2023-01-25 00:00:00-06:00	26.1475	2.39680135	0:00:00	1 day, 0:00:00

2023-01-25 00:00:00-06:00	2023-01-26 00:00:00-06:00	34.9	3.124541125	0:00:00	1 day, 0:00:00

2023-01-26 00:00:00-06:00	2023-01-27 00:00:00-06:00	25.605	2.37536605	0:00:00	1 day, 0:00:00

2023-01-27 00:00:00-06:00	2023-01-28 00:00:00-06:00	30.2125	2.486395825	0:00:00	1 day, 0:00:00

2023-01-28 00:00:00-06:00	2023-01-29 00:00:00-06:00	28.9925	2.50918945	0:00:00	1 day, 0:00:00

2023-01-29 00:00:00-06:00	2023-01-30 00:00:00-06:00	26.84	2.352525275	0:00:00	1 day, 0:00:00

2023-01-30 00:00:00-06:00	2023-01-31 00:00:00-06:00	33.9575	3.1569132	0:00:00	1 day, 0:00:00

2023-01-31 00:00:00-06:00	2023-02-01 00:00:00-06:00	33.1475	3.23518315	0:00:00	1 day, 0:00:00

2023-02-01 00:00:00-06:00	2023-02-02 00:00:00-06:00	30.65	2.835511375	0:00:00	1 day, 0:00:00

2023-02-02 00:00:00-06:00	2023-02-03 00:00:00-06:00	26.235	2.22147565	0:00:00	1 day, 0:00:00

2023-02-03 00:00:00-06:00	2023-02-04 00:00:00-06:00	38.46	3.433606275	0:00:00	1 day, 0:00:00

2023-02-04 00:00:00-06:00	2023-02-05 00:00:00-06:00	29.635	2.4040409	0:00:00	1 day, 0:00:00

2023-02-05 00:00:00-06:00	2023-02-06 00:00:00-06:00	24.015	1.987782425	0:00:00	1 day, 0:00:00

2023-02-06 00:00:00-06:00	2023-02-07 00:00:00-06:00	36.415	3.08948345	0:00:00	1 day, 0:00:00

2023-02-07 00:00:00-06:00	2023-02-08 00:00:00-06:00	30.1375	2.520288825	0:00:00	1 day, 0:00:00

2023-02-08 00:00:00-06:00	2023-02-09 00:00:00-06:00	28.425	2.48002685	0:00:00	1 day, 0:00:00

2023-02-09 00:00:00-06:00	2023-02-10 00:00:00-06:00	29.945	2.521370325	0:00:00	1 day, 0:00:00

2023-02-10 00:00:00-06:00	2023-02-11 00:00:00-06:00	30.6175	2.545089425	0:00:00	1 day, 0:00:00

2023-02-11 00:00:00-06:00	2023-02-12 00:00:00-06:00	23.1925	1.936913725	0:00:00	1 day, 0:00:00

2023-02-12 00:00:00-06:00	2023-02-13 00:00:00-06:00	16.815	1.439623075	0:00:00	1 day, 0:00:00

2023-02-13 00:00:00-06:00	2023-02-14 00:00:00-06:00	24.5225	2.086496675	0:00:00	1 day, 0:00:00

2023-02-14 00:00:00-06:00	2023-02-15 00:00:00-06:00	24.565	1.990961875	0:00:00	1 day, 0:00:00

2023-02-15 00:00:00-06:00	2023-02-16 00:00:00-06:00	30.2725	2.655019775	0:00:00	1 day, 0:00:00

2023-02-16 00:00:00-06:00	2023-02-17 00:00:00-06:00	29.025	2.792119325	0:00:00	1 day, 0:00:00

2023-02-17 00:00:00-06:00	2023-02-18 00:00:00-06:00	30.1525	2.911893425	0:00:00	1 day, 0:00:00

2023-02-18 00:00:00-06:00	2023-02-19 00:00:00-06:00	23.6075	1.86678915	0:00:00	1 day, 0:00:00

2023-02-19 00:00:00-06:00	2023-02-20 00:00:00-06:00	13.82	1.0698706	0:00:00	1 day, 0:00:00

2023-02-20 00:00:00-06:00	2023-02-21 00:00:00-06:00	33.2675	2.888854125	0:00:00	1 day, 0:00:00

2023-02-21 00:00:00-06:00	2023-02-22 00:00:00-06:00	29.4825	2.43592425	0:00:00	1 day, 0:00:00

2023-02-22 00:00:00-06:00	2023-02-23 00:00:00-06:00	29.3575	2.6474878	0:00:00	1 day, 0:00:00

2023-02-23 00:00:00-06:00	2023-02-24 00:00:00-06:00	37.54	3.15816995	0:00:00	1 day, 0:00:00

2023-02-24 00:00:00-06:00	2023-02-25 00:00:00-06:00	31.11	2.71141405	0:00:00	1 day, 0:00:00

2023-02-25 00:00:00-06:00	2023-02-26 00:00:00-06:00	13.7875	1.2186579	0:00:00	1 day, 0:00:00

2023-02-26 00:00:00-06:00	2023-02-27 00:00:00-06:00	13.62	1.119590325	0:00:00	1 day, 0:00:00

2023-02-27 00:00:00-06:00	2023-02-28 00:00:00-06:00	43.655	3.46832725	0:00:00	1 day, 0:00:00

2023-02-28 00:00:00-06:00	2023-03-01 00:00:00-06:00	39.095	3.550899725	0:00:00	1 day, 0:00:00

2023-03-01 00:00:00-06:00	2023-03-02 00:00:00-06:00	24.9425	2.156150875	0:00:00	1 day, 0:00:00

2023-03-02 00:00:00-06:00	2023-03-03 00:00:00-06:00	29.665	2.88610135	0:00:00	1 day, 0:00:00

2023-03-03 00:00:00-06:00	2023-03-04 00:00:00-06:00	21.8275	2.252724925	0:00:00	1 day, 0:00:00

2023-03-04 00:00:00-06:00	2023-03-05 00:00:00-06:00	23.81	2.1958558	0:00:00	1 day, 0:00:00

2023-03-05 00:00:00-06:00	2023-03-06 00:00:00-06:00	19.9425	1.654511275	0:00:00	1 day, 0:00:00

2023-03-06 00:00:00-06:00	2023-03-07 00:00:00-06:00	28.05	2.56680775	0:00:00	1 day, 0:00:00

2023-03-07 00:00:00-06:00	2023-03-08 00:00:00-06:00	23.1575	1.9953014	0:00:00	1 day, 0:00:00

2023-03-08 00:00:00-06:00	2023-03-09 00:00:00-06:00	27.77	2.430727	0:00:00	1 day, 0:00:00

2023-03-09 00:00:00-06:00	2023-03-10 00:00:00-06:00	25.0575	2.24815015	0:00:00	1 day, 0:00:00

2023-03-10 00:00:00-06:00	2023-03-11 00:00:00-06:00	27.81	2.5086128	0:00:00	1 day, 0:00:00

2023-03-11 00:00:00-06:00	2023-03-12 00:00:00-06:00	18.685	1.617906075	0:00:00	1 day, 0:00:00

2023-03-12 00:00:00-06:00	2023-03-13 00:00:00-05:00	32.9625	2.75841355	0:00:00	23:00:00

2023-03-13 00:00:00-05:00	2023-03-14 00:00:00-05:00	31.2025	2.617630725	0:00:00	1 day, 0:00:00
`max_req = 127` 
/Users/brett/.virtualenvs/opower-quohth9L/bin/python /Users/brett/pcode/opower/src/demo.py --utility comed --username [email protected] --password XXX --start_date 2022-10-13T00:00:00-05:00 --end_date 2023-03-12T00:00:00-06:00 --aggregate_type day --verbose

DEBUG:/Users/brett/pcode/opower/src/opower/opower.py:Fetching: https://cec.opower.com/ei/edge/apis/multi-account-v1/cws/cec/customers?offset=0&batchSize=100&addressFilter=

DEBUG:/Users/brett/pcode/opower/src/opower/opower.py:Fetching: https://cec.opower.com/ei/edge/apis/bill-forecast-cws-v1/cws/cec/customers/9e967f23-9e95-11eb-bf8b-0200170058ac/combined-forecast


Current bill forecast: Forecast(account=Account(customer=Customer(uuid='9e967f23-9e95-11eb-bf8b-0200170058ac'), uuid='a3ba62b5-9e95-11eb-bf8b-0200170058ac', utility_account_id='0855043257', meter_type=<MeterType.ELEC: 'ELEC'>, read_resolution=None), start_date=datetime.date(2023, 8, 1), end_date=datetime.date(2023, 8, 30), current_date=datetime.date(2023, 8, 10), unit_of_measure=<UnitOfMeasure.KWH: 'KWH'>, usage_to_date=309.0, cost_to_date=0.0, forecasted_usage=1337.0, forecasted_cost=0.0, typical_usage=1099.0, typical_cost=0.0)


Getting historical data: account= Account(customer=Customer(uuid='9e967f23-9e95-11eb-bf8b-0200170058ac'), uuid='a3ba62b5-9e95-11eb-bf8b-0200170058ac', utility_account_id='0855043257', meter_type=<MeterType.ELEC: 'ELEC'>, read_resolution=<ReadResolution.HALF_HOUR: 'HALF_HOUR'>) aggregate_type= day start_date= 2022-10-13 00:00:00-05:00 end_date= 2023-03-12 00:00:00-06:00

DEBUG:/Users/brett/pcode/opower/src/opower/opower.py:Fetching: https://cec.opower.com/ei/edge/apis/DataBrowser-v1/cws/cost/utilityAccount/a3ba62b5-9e95-11eb-bf8b-0200170058ac?aggregateType=day&startDate=2022-11-06T00%3A00%3A00-05%3A00&endDate=2023-03-13T00%3A00%3A00-05%3A00

Traceback (most recent call last):

File "/Users/brett/pcode/opower/src/demo.py", line 151, in 

asyncio.run(_main())

File "/Users/brett/.pyenv/versions/3.10.1/lib/python3.10/asyncio/runners.py", line 44, in run

return loop.run_until_complete(main)

File "/Users/brett/.pyenv/versions/3.10.1/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete

return future.result()

File "/Users/brett/pcode/opower/src/demo.py", line 121, in _main

cost_data = await opower.async_get_cost_reads(

File "/Users/brett/pcode/opower/src/opower/opower.py", line 327, in async_get_cost_reads

reads = await self._async_get_dated_data(

File "/Users/brett/pcode/opower/src/opower/opower.py", line 431, in _async_get_dated_data

reads = await self._async_fetch(

File "/Users/brett/pcode/opower/src/opower/opower.py", line 475, in _async_fetch

raise err

File "/Users/brett/pcode/opower/src/opower/opower.py", line 463, in _async_fetch

async with self.session.get(

File "/Users/brett/.virtualenvs/opower-quohth9L/lib/python3.10/site-packages/aiohttp/client.py", line 1141, in aenter

self._resp = await self._coro

File "/Users/brett/.virtualenvs/opower-quohth9L/lib/python3.10/site-packages/aiohttp/client.py", line 643, in _request

resp.raise_for_status()

File "/Users/brett/.virtualenvs/opower-quohth9L/lib/python3.10/site-packages/aiohttp/client_reqrep.py", line 1005, in raise_for_status

raise ClientResponseError(

aiohttp.client_exceptions.ClientResponseError: 500, message='Internal Server Error', url=URL('https://cec.opower.com/ei/edge/apis/DataBrowser-v1/cws/cost/utilityAccount/a3ba62b5-9e95-11eb-bf8b-0200170058ac?aggregateType=day&startDate=2022-11-06T00:00:00-05:00&endDate=2023-03-13T00:00:00-05:00')

I notice that the first request in each of these has the startDate greater than the date passed in as a parameter to the script.

@tronikos
Copy link
Owner

The code fetches batches backwards see https://github.com/tronikos/opower/blob/main/src/opower/opower.py#L420 so that it can terminate early when there is no more data in the past. For my PG&E account, daily and hourly data is only available for the past 3 years. If the provided date range was 10 years and since, at least for PG&E, I can request data in ~1 year ranges, going backwards we only have to make 4 requests. If we were going forward we would have to make 10 requests.

@tronikos tronikos added the bug Something isn't working label Aug 14, 2023
@tronikos
Copy link
Owner

It's strange that the max_request_days depends on the start and end date. Can you try with --aggregate_type bill and then experiment with different start and end dates and see if there is any correlation to the bill start and end dates? On my utility website opower always requests daily aggregated data between the bill range, ~30 days, and hourly aggregated data within a single day. By trial and error I set the current values of max_request_days in an attempt to limit outgoing HTTP requests.

@tronikos tronikos mentioned this issue Sep 18, 2023
Open
@warren1201
Copy link

I am also having an issue with Comed. See below. I am not a programmer so I can't really tell why. The two factor is turned off and there is no other option.

Logger: homeassistant.components.opower.coordinator
Source: helpers/update_coordinator.py:290
Integration: Opower (documentation, issues)
First occurred: November 1, 2023 at 11:47:41 PM (3 occurrences)
Last logged: 2:03:25 AM

Unexpected error fetching Opower data: 400, message='Bad Request', url=URL('https://secure.comed.com/api/Services/OpowerService.svc/GetOpowerToken')
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/opower/opower.py", line 193, in async_login
self.access_token = await self.utility.async_login(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/opower/utilities/exelon.py", line 209, in async_login
async with session.post(
File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 1141, in aenter
self._resp = await self._coro
^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 643, in _request
resp.raise_for_status()
File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1005, in raise_for_status
raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 400, message='Bad Request', url=URL('https://secure.comed.com/api/Services/OpowerService.svc/GetOpowerToken')

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 290, in _async_refresh
self.data = await self._async_update_data()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/opower/coordinator.py", line 79, in _async_update_data
await self.api.async_login()
File "/usr/local/lib/python3.11/site-packages/opower/opower.py", line 201, in async_login
raise CannotConnect(err)
opower.exceptions.CannotConnect: 400, message='Bad Request', url=URL('https://secure.comed.com/api/Services/OpowerService.svc/GetOpowerToken')

@tronikos
Copy link
Owner

tronikos commented Nov 5, 2023

@warren1201 your issue is different than this. Yours fail while getting the Opower access token. Most likely the request now requires Referer similar to #48
Without access to a ComEd account I unfortunately cannot fix this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@bjackson @tronikos @warren1201