Skip to content

Conversation

andlaus
Copy link
Member

@andlaus andlaus commented May 20, 2025

This ODX category is used to hold variant coding information. Just like for the FLASH category, the PR is relatively large, but it does not contain much logic on the odxtools level.

Andreas Lauser <andreas.lauser@mercedes-benz.com>, on behalf of MBition GmbH.
Provider Information

andlaus added 4 commits May 20, 2025 13:56
this is used for variant coding.

Signed-off-by: Andreas Lauser <andreas.lauser@mercedes-benz.com>
Approved-by: Christian Hackenbeck <christian.hackenbeck@mercedes-benz.com>
whereas issues == "non-spec compliant behaviour in .odx-e files". That
said, to me it rather seems like these two issues are mistakes in the
spec...

Signed-off-by: Andreas Lauser <andreas.lauser@mercedes-benz.com>
Approved-by: Christian Hackenbeck <christian.hackenbeck@mercedes-benz.com>
Signed-off-by: Andreas Lauser <andreas.lauser@mercedes-benz.com>
Approved-by: Christian Hackenbeck <christian.hackenbeck@mercedes-benz.com>
Signed-off-by: Andreas Lauser <andreas.lauser@mercedes-benz.com>
Approved-by: Christian Hackenbeck <christian.hackenbeck@mercedes-benz.com>
@andlaus andlaus requested a review from kayoub5 May 20, 2025 11:57
for sdg in self.sdgs:
result.update(sdg._build_odxlinks())

return result
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit off topic, but maybe _build_odxlinks, _resolve_odxlinks and _resolve_snrefs, can be simplified but have one function called _get_referenceable(self) -> list[Optional[LinkableElement]], that return all the objects that should have _build_odxlinks, _resolve_odxlinks and _resolve_snrefs called on them, the rest of the logic can be implemented in the base class.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the "leafs" of _build_odxlinks() should be moved into IdentifiableElement (#425). For resolving the references, I don't really see a good alternative to the current solution. Maybe we can introduce a proxy class like weakref.proxy but I'm not sure if this such a terribly good idea as it is quite "magical"...


# at most one of the following two attributes is not None
datafile: Datafile | None = None
data: str | None = None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add helper method that returns actual data from either data or datafile ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea. this lead to me falling into a pretty deep rabbit hole about some arcane file formats for specifying flash blobs (Intel-HEX and S-RECORD/Motorola-S): 0f46228

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

@andlaus andlaus Jun 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not so sure if having a additional non-trivial dependency is a good idea (this can quickly degenerate into dependency hell like most NPM and Java projects). also, hexrec seems to have problems with one of the datasets which have been used internally at our company for ages (it raises a ValueError('non-contiguous data within range') exception). bincopy can handle the dataset, but it fills these gaps in the binary blob with 0xff bytes instead of 0x00. (That said, I'm not sure how "gaps" are supposed to be filled or if it even matters. I guess that using zeros might be advantageous, because the memory controller of the ECU's microchip might compress all-zero pages by pointing it to the same memory location...)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, if one needs to have more control when handling these datasets, one can always feed the "raw" data to the library of choice...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, I've looked this up more closely: it seems like using 0xff as the fill bytes for gaps indeed seems to be the de-facto standard for the SREC and intel-hex formats. I thus changed the code to use the bincopy module.

@andlaus
Copy link
Member Author

andlaus commented Jun 6, 2025

I made the CI pass again: seems like mypy 1.16 was recently released and it looks like it is quite a bit pickier with bytes vs bytearray than 1.15. (IIRC the old policy was to treat them as basically equal.)

@kayoub5: do you see any showstoppers left in this PR?

andlaus added 3 commits June 12, 2025 09:13
i.e., deal with the nitty-gritty details of the Intel-HEX and
Motorola-S file formats...

also, provide processed versions of the values for some tags which
specify a data type (ENCRYPT-COMPRESS-METHOD, VALIDITY-FOR, etc.)

thanks to [at]kayoub5 for the nudge!

Signed-off-by: Andreas Lauser <andreas.lauser@mercedes-benz.com>
Approved-by: Katja Köhler <katja.koehler@mercedes-benz.com>
by far the biggest churn here was caused by the fact that mypy 1.16 is
much pickier about distinguishing between `bytes` and `bytearray` than
1.15 is...

Signed-off-by: Andreas Lauser <andreas.lauser@mercedes-benz.com>
this was meant to be as a to-do reminder. as [at]kayoub5 pointed out,
this is better handled using a github issue...

Signed-off-by: Andreas Lauser <andreas.lauser@mercedes-benz.com>
…-HEX data records and flash data

thanks to [at]kayoub5 for bringing this up.

Signed-off-by: Andreas Lauser <andreas.lauser@mercedes-benz.com>
@andlaus
Copy link
Member Author

andlaus commented Jun 23, 2025

@kayoub5: I switched to using the bincopy module. do you see anything which should prevent merging this PR?

(BTW: ODX' process model for flashing is quite arcane IMO. Possibly we should add some additional logic to simplify user code w.r.t. filters, segments and all that jazz? something like:

for address, blob in my_flashdata.chunks:
  write_to_ecu_flash(address, blob)

)

@andlaus andlaus merged commit 3176f44 into mercedes-benz:main Jun 25, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants