Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[weather_owm] API 2.5 is dying in June, 2024 #2246

Closed
mlmatlock opened this issue May 5, 2024 · 12 comments · Fixed by #2249
Closed

[weather_owm] API 2.5 is dying in June, 2024 #2246

mlmatlock opened this issue May 5, 2024 · 12 comments · Fixed by #2249

Comments

@mlmatlock
Copy link
Contributor

mlmatlock commented May 5, 2024

How can we help you today?

Openweathermap.org sent out information stating that API 2.5 will be killed off in June, and to migrate to API 3.0. I'm not a coder (and never played one on TV), so I'm clueless as to going about changing the module to adapt to the 3.0 API.

Your py3status version

py3status version 3.57 (python 3.12.3) on sway

Share your configuration

Config: https://0x0.st/XX0o.txt

Additional context

I've gotten a 3.0 API key. This is the full output (in imperial units) of a 3.0 call for my location (no exclusions):

Full output: https://0x0.st/XX0X.output

I ran the output thru a JSON formatter:

JSON: https://0x0.st/XX08.json

Thanks in advance for any help!!!

ETA: I'm willing to provide an API 3.0 key (via email) to help with modifying the module.

@lasers
Copy link
Contributor

lasers commented May 7, 2024

Please fill you billing information for subscription

This is unfortunate.

It looks like the users would need to start filling out personal / billing plan in order to subscribe / get new API key even although they might not want to.

The users need to know they can get charged if they made excess calls after the limit.

I ask @ultrabug what he want to do here. Possible removal just like a29ac2c.

@mlmatlock
Copy link
Contributor Author

mlmatlock commented May 7, 2024

What if it was relegated to a 'user' module (put in $XDG_CONFIG_HOME/py3status/modules)? That way, whoever wants to subscribe, can. If they don't, then no loss to them (other than it not working anymore).

I really hope I don't have to lose this...

@lasers
Copy link
Contributor

lasers commented May 8, 2024

/cc author @alexoneill

@ultrabug
Copy link
Owner

ultrabug commented May 9, 2024

It's bank holiday week here, I'll have a look next week and will do my best to keep this module up

@ultrabug
Copy link
Owner

So it seems my current (pro) subscription does not include 3.0 API :(

I'll have a deeper look ... damn

@ultrabug
Copy link
Owner

Tested new endpoint, seems like the response payload is the same

diff --git a/py3status/modules/weather_owm.py b/py3status/modules/weather_owm.py
index 8aa47dce..2721c869 100644
--- a/py3status/modules/weather_owm.py
+++ b/py3status/modules/weather_owm.py
@@ -265,7 +265,7 @@ import datetime
 # API information
 OWM_CURR_ENDPOINT = "https://api.openweathermap.org/data/2.5/weather?"
 OWM_FUTURE_ENDPOINT = "https://api.openweathermap.org/data/2.5/forecast?"
-OWM_ONECALL_ENDPOINT = "https://api.openweathermap.org/data/2.5/onecall?"
+OWM_ONECALL_ENDPOINT = "https://api.openweathermap.org/data/3.0/onecall?exclude=alerts,minutely"
 IP_ENDPOINT = "http://geo.ultrabug.fr"
 
 # Paths of information to extract from JSON

@ultrabug
Copy link
Owner

@mlmatlock could you try using
weather_owm.py.gz

and confirm it works for you as intended?

@ultrabug
Copy link
Owner

If it works, the questioning will relate to the fact that people could be charged by using this module so I really wonder if we want to add some kind of counter to make sure that the module stays within the free limit of api calls.

Since users have to enter their credit card to use that endpoint, they should be aware enough anyway so on the principle it's not the module's business.

As of now, the cache timeout is 30min which makes the 1000 calls / day limit hard to reach, but... you never know.

@mlmatlock
Copy link
Contributor Author

mlmatlock commented May 16, 2024

@mlmatlock could you try using weather_owm.py.gz

and confirm it works for you as intended?

@ultrabug, it seems to be working!

Noticed one thing, though...it's working with my old (2+ years) API key. If I generate a new API key and put that one in the config, I get a 401 error. A little digging, and if I call the 2.5 API with the new key from a web browser, I get the same 401 error:

{"cod":401, "message": "Invalid API key. Please see https://openweathermap.org/faq#error401 for more info."}

Calling the 2.5 API with the old key works, old key also calls the 3.0 API with no problems.

Seems new API keys are forcing the depreciation, and old keys might stop working when they actually turn the 2.5 API off.

As always, I appreciate the help!

@lasers
Copy link
Contributor

lasers commented May 17, 2024

Some thoughts.

As of now, the cache timeout is 30min which makes the 1000 calls / day limit hard to reach, but... you never know.

  1. If users doesn't like this, they'll change it to every 5 minutes or such... Suggestion... Add self.cache_timeout = max(600, self.cache_timeout) too as a preventive measure. The API said it updates every 10 minutes anyway.

to add some kind of counter to make sure that the module stays within the free limit of api calls.

  1. Users refreshing/restarting i3 frequently..... is a possible occurrence. Storing cache between i3-msg restarts if it is under max(600, self.cache_timeout) would be nice to have too.

  2. Option to shutdown on 999 calls?

This is given. Clean up code. Get rid of 2.5 and get everything from 3.0 will force users to refresh API key and they'll learn about this new billing subscription. No issue there. Do document the paid plan for existing/new users anyway.

I think 1) and 2) matters. 3) is overkill.

  1. Onecall 3.0 likely expose new placeholders. See if we should make format_* for some of them. Looks like we may be able to customize exclude request based on placeholders to make calls smaller too. Idk.

@ultrabug
Copy link
Owner

Thanks @lasers indeed I considered some of those options. I think even if it requires more work, spending time on 3 would be more future proof.

I implemented 1/ and 2/ anyway

diff --git a/py3status/modules/weather_owm.py b/py3status/modules/weather_owm.py
index 8aa47dce..54c80669 100644
--- a/py3status/modules/weather_owm.py
+++ b/py3status/modules/weather_owm.py
@@ -261,11 +261,12 @@ diff
 """
 
 import datetime
+import json
 
 # API information
 OWM_CURR_ENDPOINT = "https://api.openweathermap.org/data/2.5/weather?"
 OWM_FUTURE_ENDPOINT = "https://api.openweathermap.org/data/2.5/forecast?"
-OWM_ONECALL_ENDPOINT = "https://api.openweathermap.org/data/2.5/onecall?"
+OWM_ONECALL_ENDPOINT = "https://api.openweathermap.org/data/3.0/onecall?exclude=alerts,minutely"
 IP_ENDPOINT = "http://geo.ultrabug.fr"
 
 # Paths of information to extract from JSON
@@ -448,6 +449,18 @@ class Py3status:
         # Generate our icon array
         self.icons = self._get_icons()
 
+        # Implement safe-to-reload rate limit
+        cached_hour = datetime.datetime.now(datetime.UTC).strftime("%H")
+        self.cached_hits = json.loads(
+            self.py3.storage_get("cached_hits") or json.dumps({cached_hour: 0})
+        )
+        self.cached_onecall_response = self.py3.storage_get("cached_onecall_response")
+
+        # We want to make sure users to not exceed the request limit
+        # to 3.0 API and get billed while taking into account that
+        # OWM does refresh its API data every 10min anyway.
+        self.cache_timeout = max(600, self.cache_timeout)
+
         # Verify the units configuration
         if self.unit_rain.lower() not in RAIN_UNITS:
             raise Exception("unit_rain is not recognized")
@@ -817,6 +830,9 @@ class Py3status:
         return self.py3.safe_format(self.format, today)
 
     def weather_owm(self):
+        # Prepare rate limit cache hour
+        cached_hour = datetime.datetime.now(datetime.UTC).strftime("%H")
+        cached_hits = self.cached_hits.get(cached_hour, 0)
         # Get weather information
         loc_tz_info = self._get_loc_tz_info()
         text = ""
@@ -840,9 +856,19 @@ class Py3status:
             except Exception:
                 raise Exception("no latitude/longitude found for your config")
 
-            # onecall = forecasts
-            onecall_api_params = {"lat": lat, "lon": lon}
-            onecall = self._get_onecall(onecall_api_params)
+            # onecall = forecasts rate limited
+            if cached_hits < 999:
+                onecall_api_params = {"lat": lat, "lon": lon}
+                onecall = self._get_onecall(onecall_api_params)
+                # update and store caches
+                self.cached_onecall_response = onecall
+                self.cached_hits[cached_hour] = cached_hits + 1
+                self.py3.storage_set("cached_onecall_response", onecall)
+                self.py3.storage_set(
+                    "cached_hits", json.dumps({cached_hour: self.cached_hits[cached_hour]})
+                )
+            else:
+                onecall = self.cached_onecall_response
             onecall_daily = onecall["daily"]
 
             fcsts_days = self.forecast_days + 1

@ultrabug
Copy link
Owner

PR is here: #2249

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants