[CDG5] Random questions

[Elliot Nunn]

I have prepared macresources for 'proper' compression support in a new
branch.

Currently the actual decompression/recompression is faked out by this
file. Max, is it possible for you to wrap your code in a Python package
(folder with an __init__.py) that exports a similar API, for inclusion
in the macresources repo?

https://github.com/elliotnunn/macresources/blob/compression/macresources/FakeResDecompress.py

Regrettably the System 7 resource compression scheme was a bit messy.
Additionally, Rez and DeRez treat compressed resources a bit oddly
because those tools do not call the Resource Manager (for 'historical'
reasons, likely the Lisa). These two constraints have forced me to
implement a quite complicated algorithm for parsing and representing
compressed resources. I plan to expose this code path via the '-expand'
argument to SimpleDeRez. Currently it does these things:

- Exposes decompressed data via the Resource.data attribute. If the data
  is in an unsupported format the access fails unless the user sets
  Resource.expand_err_ok. This error is also suppressed when trying to
  extract a resource for saving to a Rez file, because in that case the
  Rez code generator will cancel the decompression and leave the
  'compressed' flag set.

- Lazily decompresses and recompresses a resource only when absolutely
  necessary.

- No matter what, Rez or SimpleRez will always be able to slurp a file I
  have generated, and produce a resource fork that contains correct data
  when accessed via the Resource Manager (although it may no longer be
  compressed). SimpleRez will recognise commented annotations like /*
  Compress: DonnBits */ and do the needful, while Rez will naturally
  ignore them.

- It is always safe for an API user to change the
  Resource.compression_format attribute, say to 'DonnBits' or to None.
  Care is taken that while binary data can be *converted* between
  formats, it will never be *interpreted* in the wrong format.

To clarify the first point a bit, here are some of the Rez-language
outputs that might be produced when extraction is attempted:

- data 'abcd' (1, $01) /* Unknown compression format */

  ... followed by compressed data (note the compressed bit is set in
  $01)

- data 'abcd' (1) /* Compress: DonnBits */

  ... followed by expanded data (which SimpleRez will recompress, but
  but Rez won't)

By the way, I think it was Jason who first got us onto the topic of
compressed resources. Good thinking man! This is an idea whose time has
come, because I have some plans for the System size of the CubeE repo,
compressed resources and all.

Elliot

> On 10 Dec 2018, at 11:10 pm, Elliot Nunn <elliotnunn at fastmail.com> wrote:
> 
> Grepping through System 9.2.2 reveals several compressed resources:
> 
> $ SimpleDeRez ~/System/..namedfork/rsrc -compcmt -ascii | grep -A3 _compressed | head -n9
> data 'CDEF' (62, $61 /*sysheap, purgeable, _compressed*/) {
> 	$"A89F 6572 0012 0901 0000 0454 0003 03FF"            /* ..er.......T.... */
> 	$"0000 3F04 E56F FF44 8E70 7082 A2E0 0083"            /* ..?..o.D.pp..... */
> 	$"E2E0 0122 86E0 00E7 C0A2 0544 3EEF FF85"            /* ...".......D>... */
> --
> data 'DRVR' (2, ".Print", $71 /*sysheap, purgeable, locked, _compressed*/) {
> 	$"A89F 6572 0012 0901 0000 019A 0003 03FF"            /* ..er............ */
> 	$"0000 2278 058F FC90 00A0 00B0 00C0 00D0"            /* .."x............ */
> 	$"18B9 41C9 A5B9 D080 0001 1149 5948 0008"            /* ..A........IYH.. */
> 
> Could you have used a search method that extracted the resources
> automatically?
> 
> Here are some Donn resources, both compressed and extracted (manual
> operation... ugh!). How will I identify the mysterious Mac OS 9 format?
> 
> https://github.com/elliotnunn/donn-examples
> 
> Good for Greg!
> 
>> On 10 Dec 2018, at 9:24 pm, Max Poliakovski <maximumspatium at googlemail.com> wrote:
>> 
>> MacOS 9 doesn't apparently use compressed resources. I looked into the resource map of the System file from MacOS 9.2.2 and couldn't find any resource starting with 0xA89F6572.
>> 
>> Moreover, dcmp 3 does exist as 68k code only while Donn&Gregg decompression methods have PPC equivalents in 'ncmp' resources.
>> 
>> I therefore need more compressed resources for testing purpose. It would be great I you could locate Donn-compressed resources as well. They usually start with 0xA89F6572 00XX 0801...
>> 
>> Regarding re-compression, I think it should be easy to build a compressor for GreggyBits controlled by the decompression params. I'm not sure if the original compressor (whose source isn't publicly available) had an automatic way of determining best-fit compression params...
>> 
>> BTW, the author of GreggyBits compression - Greg Marriott - still does software development. According to his blog http://www.greggybits.com/about.html, he works for Apple again. He'd be surely pleased to see his ol' stuff reimplemented in Python. I'd do in his place...
>> _______________________________________________
>> cdg5 mailing list
>> cdg5 at ucc.asn.au
>> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5
>