Skip to content

Updates the sdss mwmStar loader to fix bug with file datasums #1253

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jul 29, 2025

Conversation

havok2063
Copy link
Contributor

This PR updates the SDSS-V mwm data loaders to account for a bug in a subset of the DR19 SDSS-V mwmStar files. A subset of files have extensions with empty data arrays and non-zero DATASUM keywords. This was due to some data needing to be cleaned out but the header wasn't fully scrubbed/corrected.

The previous loader would attempt to load first extension found with non-zero DATASUM and would crash on the empty data. This fix allows the loader to correctly skip over it and load the first real data extension it finds.

It would be great if this can be backported to specutils 1.x, so it can be used currently in Jdaviz.

@rosteen
Copy link
Contributor

rosteen commented Jul 23, 2025

Thanks for the contribution @havok2063, unfortunately it looks like there's a real test failure. Could you address that, please? Once we get this in I can backport and do a bugfix release, although with the next Jdaviz release we'll be using specutils 2 there so it won't be needed for too long.

@havok2063
Copy link
Contributor Author

havok2063 commented Jul 23, 2025

@rosteen Are you referring to the Python 3.12 Windows run? Unfortunately, I don't have a windows machine to test on. I'll see if I can run the action locally. I just pushed fixes for the Code style checks. Do I need to worry about the rtd or release action failures? (looks like they are passing now)

@rosteen
Copy link
Contributor

rosteen commented Jul 23, 2025

@rosteen Are you referring to the Python 3.12 Windows run? Unfortunately, I don't have a windows machine to test on. I'll see if I can run the action locally. I just pushed fixes for the Code style checks. Do I need to worry about the rtd or release action failures? (looks like they are passing now)

Yup, I was talking about the Windows run, looks like the others were transient problems with remote data. I have a Windows machine at home I can try debugging on tomorrow or Friday, unfortunately I just went into the office for the rest of the day today.

@havok2063
Copy link
Contributor Author

Thanks for trying to look into this. I couldn't run the action locally, but I can try to spin up a Virtual Box of windows or something. I do have an old windows machine lying around that I can try to spin up git+conda on. Let me know if you end up testing this though.

@rosteen
Copy link
Contributor

rosteen commented Jul 28, 2025

I was able to reproduce the test failure locally, it's due to arr.itemsize being 0 at https://github.com/astropy/astropy/blob/412db1e64cca4e4d05cd373dc3cbe12ff2d6a257/astropy/io/fits/util.py#L589. That code is working around a bug but is 11 years old, so I wonder if its still relevant. You also may be able to change the test to work around it but I don't 100% understand the datasum issue to change it myself. I'm trying to figure out on my Windows machine if that fits code is even needed anymore...

Copy link

codecov bot commented Jul 29, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.83%. Comparing base (702f0ce) to head (66bc8c3).
⚠️ Report is 9 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1253   +/-   ##
=======================================
  Coverage   86.83%   86.83%           
=======================================
  Files          63       63           
  Lines        4732     4732           
=======================================
  Hits         4109     4109           
  Misses        623      623           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@rosteen rosteen merged commit f63271c into astropy:main Jul 29, 2025
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants