• 0 Posts
  • 28 Comments
Joined 1 year ago
cake
Cake day: June 6th, 2023

help-circle





  • I haven’t seen the rate limitation issue for a couple of months now. The issue is not new, IIRC the Aurora team needs to provide more and more anonymous accounts as the user base is growing. Some months ago we had this problem for more than two weeks. I just checked and Aurora finds no connection, but until today everything worked flawlessly. And I expect it to work again soon.

    I feel your pain though. I am degoogled for more than 5 years now and what I learned is that Google will always look for ways to make our life harder. More and more basic functions such as network location were seperated from the AOSP into proprietary google services. I am pretty sure this will get even worse in the future.

    We must not forget that we rely on open source software and a hand full if developers, everything can break tomorrow and we are fucked.


  • I think it’s a good thing polars developers are heading toward interoperability. The Dataframe Interchange Protocol the article mentions sounds interesting.

    For example, if you read the documentation for Plotly Express

    I know this seems to be an important topic in the community. But honestly, I rarely use all the plotting backends at all. They are nice for quick visualizations, but most of the time I prefer to throw my data into matplotlib on my own, just for the sake of customization.

    polars.DataFrame.to_pandas() by default uses NumPy arrays, so it will have to convert all your data from Arrow to Numpy; this will double your memory usage at least, and take some computation too. If you use Arrow, however, the conversion will take essentially no time and no extra memory is needed (“zero copy”)

    I don’t want to complain, it is definitely a good thing polars developers address this. pandas is the standard and as long as full interoperability between polars and the pandas ecosystem is lacking, this “hack” is needed. However, data transformation can be an incredibly sensitive topic. I do not even trust pandas or tensorflow in always doing the right thing when converting data - processing data in polars, converting it to pandas and then process it further - I am sceptical. And I am not even talking about performance here.

    If you’re doing heavy geographical work, there will likely someday be a replacement for GeoPandas, but for now you probably going to spend a lot of time using Pandas

    This is important. Geopandas is one of the most import libraries derived from pandas and widely used in the geoscience community. The idea of an equivalent like “geopolars” is insane in my eyes. I am biased as a data scientist mostly working on spatial data, but this is the main reason that I watch the development of polars only from the sidelines. Even if I wouldn’t work with geographic data, GeoAI is such an important topic you can’t just ignore it. And that’s only the perspective from my field, who knows what other important communities are out there that rely on pandas.




  • Absolutely understandable. Maybe there is some easy tool around, but I see some potential problems.

    Questions such as “Is this zone habitable in 2035” or “Will this area be ocean in 2050” are extremely hard to answer with our current knowledge and available data. As you probably know, climate scientists speak of probabilities, as projections e.g.for temperature are highly uncertain, especially in the more far away future. If there is a tool answering such questions, you probably can’t trust it. Then, you will likely not get a one does it all tool, as the questions OP asked are highly specific. All in all the matter is very complex and there are no easy answer. You need some kind of motivation to gather a certain amount of background knowledge about the topic.

    I think what comes closest to what OP wants is downloading model results from largely accepted climate models such as CMIP6. They usually come in special file formats that can efficiently store geospatial time series, such as netCDF or HDF5. There are tools like Panoply where you can do some very nice visualisations. You do not need to code neither is the software very complex. QGIS and ArcGIS are overkill here, as you would not want to do spatial analysis but only visualize.

    The work you would need to do is 1) understanding what you want - there is not a single result, instead you have climate projections under several different scenarios, model assumptions, input data etc. You need to figure out what to choose. 2) Have a decent feeling of geospatial visualization techniques. Cartography is a complex field, and correctly visualizing data is pretty hard.

    I am sorry I cannot provide easy solution. WhatI can offer is helping to acquire data if you what you want and also I can give technical support on visualization software. Maybe also give you some guidelines on how to interpret a figure.











  • I don’t see why not. However I like to have my constants module contain data structures as simple as possible, therefore I would prefer strings (which are also immutable), and create a Path object when I need it. I can see though that you usually won’t have the case where you would not want the path to be a Path object.