From elliotnunn at fastmail.com Sat Nov 17 22:55:30 2018 From: elliotnunn at fastmail.com (Elliot Nunn) Date: Sat, 17 Nov 2018 22:55:30 +0800 Subject: [CDG5] CubeE status Message-ID: <0BBA2D2D-C537-4A0B-9E75-2B5929F78389@fastmail.com> Hi all, When I last put serious effort into the CubeE tree about a year ago, we got it to the point where about 30% of the ~600k main code image could be built from source, with the remainder coming from "unlinked" object files of variable quality (https://github.com/elliotnunn/unlink). All of this source was assembly, because I was unable to get more than a handful of C functions to round-trip with the included MPW C 3.2.1. While the nitty-gritty of building the tree was fairly doable (git filters, empw), it was still moderately prone to breakage (type and creator codes especially) and likely to get much worse in the future (resource forks being deprecated, GetFile/SetFile/Rez/DeRez going away, modern-macOS-only). It represented a cool technical achievement, but one with little room for incremental improvement. (This became apparent when I tried to produce a Classic ROM branch.) So without further ado, please welcome the new CubeE source and build script: https://github.com/elliotnunn/CubeE https://github.com/elliotnunn/BuildCubeE I have neatened up both the original source tree (removed gremlins, converted to LF and Mac Roman) and the changes that I made in the mac-rom repo. I am currently trying to weave these into a meaningful and realistic commit history, so expect frequent rebases, and especially for the "JUNK" commit to change. Most excitingly, I have found the precise compiler version that Apple used to build their ROMs: MPW C 3.3.3, the final release, from ETO 14. With this version I have gotten the ROM Font Manager, weighing in at a respectable 88 KB and mostly written in C, to round trip perfectly. That and some other improvements bring the from-source fraction up to 45%, with plenty of work still to be done. You already know about the other tooling that I have created to get this working, but I don't think I mentioned their PyPI pages: https://pypi.org/project/machfs/ https://pypi.org/project/macresources/ Cheers, Elliot From danielbj314 at verizon.net Sun Nov 18 00:33:00 2018 From: danielbj314 at verizon.net (Daniel B-J) Date: Sat, 17 Nov 2018 11:33:00 -0500 Subject: [CDG5] CubeE status In-Reply-To: <0BBA2D2D-C537-4A0B-9E75-2B5929F78389@fastmail.com> References: <0BBA2D2D-C537-4A0B-9E75-2B5929F78389@fastmail.com> Message-ID: <6A66F548-8937-43C1-9AD5-7B6EC73131E9@verizon.net> Cool. It currently doesn't build because some of the .rdump files use a notation that macresources is unfamiliar with. An example is the file BuildCubeE/BootableMPW/System.rdump. Here is a line it doesn't like: b"data 'snth' (11, $71 /*sysheap, purgeable, locked, _compressed*/) { the parse_rez_code function is macresources understands neither the hexadecimal notation for resource attributes, nor comments inside the parenthesis. The 'args' variable inside it ends up in a weird state: [('nonstring', bytearray(b'11')), ('nonstring', bytearray(b'$71')), ('nonstring', bytearray(b'/*sysheap')), ('nonstring', bytearray(b'purgeable')), ('nonstring', bytearray(b'locked')), ('nonstring', bytearray(b'_compressed*/'))] This causes a AttributeError when the code attempts to do this: rsrcattrs |= getattr(ResourceAttrs, arg.decode('ascii')) There is no ResourceAttrs attribute '$71'. -------------- next part -------------- An HTML attachment was scrubbed... URL: https://lists.ucc.gu.uwa.edu.au/pipermail/cdg5/attachments/20181117/ad4c2fbc/attachment.htm From elliotnunn at fastmail.com Sun Nov 18 00:39:25 2018 From: elliotnunn at fastmail.com (Elliot Nunn) Date: Sun, 18 Nov 2018 00:39:25 +0800 Subject: [CDG5] CubeE status In-Reply-To: <6A66F548-8937-43C1-9AD5-7B6EC73131E9@verizon.net> References: <0BBA2D2D-C537-4A0B-9E75-2B5929F78389@fastmail.com> <6A66F548-8937-43C1-9AD5-7B6EC73131E9@verizon.net> Message-ID: <92A69E21-F132-4677-878D-D7D1D001E876@fastmail.com> Many apologies! Update your macresources and it should be fine. The commit that adds support for compressed resources is bda55a5. > On 18 Nov 2018, at 12:33 am, Daniel B-J wrote: > > Cool. > > It currently doesn't build because some of the .rdump files use a notation that macresources is unfamiliar with. > > An example is the file BuildCubeE/BootableMPW/System.rdump. Here is a line it doesn't like: > b"data 'snth' (11, $71 /*sysheap, purgeable, locked, _compressed*/) { > > the parse_rez_code function is macresources understands neither the hexadecimal notation for resource attributes, nor comments inside the parenthesis. The 'args' variable inside it ends up in a weird state: > [('nonstring', bytearray(b'11')), ('nonstring', bytearray(b'$71')), ('nonstring', bytearray(b'/*sysheap')), ('nonstring', bytearray(b'purgeable')), ('nonstring', bytearray(b'locked')), ('nonstring', bytearray(b'_compressed*/'))] > > This causes a AttributeError when the code attempts to do this: > rsrcattrs |= getattr(ResourceAttrs, arg.decode('ascii')) > > There is no ResourceAttrs attribute '$71'. > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From danielbj314 at verizon.net Sun Nov 18 08:14:53 2018 From: danielbj314 at verizon.net (Daniel B-J) Date: Sat, 17 Nov 2018 19:14:53 -0500 Subject: [CDG5] CubeE status In-Reply-To: <92A69E21-F132-4677-878D-D7D1D001E876@fastmail.com> References: <0BBA2D2D-C537-4A0B-9E75-2B5929F78389@fastmail.com> <6A66F548-8937-43C1-9AD5-7B6EC73131E9@verizon.net> <92A69E21-F132-4677-878D-D7D1D001E876@fastmail.com> Message-ID: What MPW commands should I use to build CubeE? That Make:Build script is very large and very complicated. From elliotnunn at fastmail.com Sun Nov 18 08:17:39 2018 From: elliotnunn at fastmail.com (Elliot Nunn) Date: Sun, 18 Nov 2018 08:17:39 +0800 Subject: [CDG5] CubeE status In-Reply-To: References: <0BBA2D2D-C537-4A0B-9E75-2B5929F78389@fastmail.com> <6A66F548-8937-43C1-9AD5-7B6EC73131E9@verizon.net> <92A69E21-F132-4677-878D-D7D1D001E876@fastmail.com> Message-ID: Yeah, it's a monster. Not sure why. Mikey actually rewrote it at one point. This works fine: :Make:Build RISC -src : > On 18 Nov 2018, at 8:14 am, Daniel B-J wrote: > > What MPW commands should I use to build CubeE? > > That Make:Build script is very large and very complicated. > > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From danielbj314 at verizon.net Sun Nov 18 08:44:49 2018 From: danielbj314 at verizon.net (Daniel B-J) Date: Sat, 17 Nov 2018 19:44:49 -0500 Subject: [CDG5] CubeE status In-Reply-To: References: <0BBA2D2D-C537-4A0B-9E75-2B5929F78389@fastmail.com> <6A66F548-8937-43C1-9AD5-7B6EC73131E9@verizon.net> <92A69E21-F132-4677-878D-D7D1D001E876@fastmail.com> Message-ID: <33E06896-6F6F-4E6C-92C5-6088152B04A1@verizon.net> > This works fine: > > :Make:Build RISC -src : Works perfect. Of course I can't really test the built ROM myself. Would BuildCubeE also work with the powermac-rom repo? Somewhat relevant feature request: can you make a tool that takes source code in a disk image or .sit file and makes it Github-friendly and build-able with BuildCubeE? What you are doing is really really useful. From elliotnunn at fastmail.com Sun Nov 18 08:56:12 2018 From: elliotnunn at fastmail.com (Elliot Nunn) Date: Sun, 18 Nov 2018 08:56:12 +0800 Subject: [CDG5] CubeE status In-Reply-To: <33E06896-6F6F-4E6C-92C5-6088152B04A1@verizon.net> References: <0BBA2D2D-C537-4A0B-9E75-2B5929F78389@fastmail.com> <6A66F548-8937-43C1-9AD5-7B6EC73131E9@verizon.net> <92A69E21-F132-4677-878D-D7D1D001E876@fastmail.com> <33E06896-6F6F-4E6C-92C5-6088152B04A1@verizon.net> Message-ID: <6DC0C71C-8E55-4406-AF39-B7C34F6DBA62@fastmail.com> Phew! Thanks for testing. Any chance you're keen to fix up the SizeMem code that I have mislabelled as "Lastly"? If you have newworld-rom working, you can just: dd if=CubeE/BuildResults/RISC/Image/RomMondo of=newworld-rom/rom conv=notrunc But soon (after third year exams, which are in a week) I will look at "porting" powermac-rom and NanoKernel to this system. If you have sources in a disk image, just run DumpHFS, a command line tool in machfs. I hope to evolve BuildCubeE into the "general solution" to MPW builds (possibly CodeWarrior too?). > On 18 Nov 2018, at 8:44 am, Daniel B-J wrote: > >> This works fine: >> >> :Make:Build RISC -src : > > Works perfect. > > Of course I can't really test the built ROM myself. Would BuildCubeE also work with the powermac-rom repo? > > Somewhat relevant feature request: can you make a tool that takes source code in a disk image or .sit file and makes it Github-friendly and build-able with BuildCubeE? What you are doing is really really useful. > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From danielbj314 at verizon.net Sun Nov 18 11:29:07 2018 From: danielbj314 at verizon.net (Daniel B-J) Date: Sat, 17 Nov 2018 22:29:07 -0500 Subject: [CDG5] CubeE status In-Reply-To: <6DC0C71C-8E55-4406-AF39-B7C34F6DBA62@fastmail.com> References: <0BBA2D2D-C537-4A0B-9E75-2B5929F78389@fastmail.com> <6A66F548-8937-43C1-9AD5-7B6EC73131E9@verizon.net> <92A69E21-F132-4677-878D-D7D1D001E876@fastmail.com> <33E06896-6F6F-4E6C-92C5-6088152B04A1@verizon.net> <6DC0C71C-8E55-4406-AF39-B7C34F6DBA62@fastmail.com> Message-ID: <9D81091C-DECC-49D0-A1E1-FA6B9F3C5F16@verizon.net> > Phew! Thanks for testing No problem. > If you have sources in a disk image, just run DumpHFS, a command line tool in > machfs. DumpHFS works great. One of my smaller test tools is now on Github: https://github.com/DBJ314/GetPMFeatures . I'll eventually stick my other projects on there as well. Can't build it using BuildCubeE, because of the lack of InterfaceLib and other important libraries. Not really a problem as BuildCubeE isn't meant for that yet. > If you have newworld-rom working, you can just: > > dd if=CubeE/BuildResults/RISC/Image/RomMondo of=newworld-rom/rom conv=notrunc Seems to work. Will test the tbxi later. > I hope to evolve BuildCubeE into the "general solution" to MPW builds > (possibly CodeWarrior too?). Cool. > . Any chance you're keen to fix up the SizeMem > code that I have mislabelled as "Lastly"? Not likely. I am still a master procrastinator. -------------- next part -------------- An HTML attachment was scrubbed... URL: https://lists.ucc.gu.uwa.edu.au/pipermail/cdg5/attachments/20181117/47dd7f43/attachment.htm From elliotnunn at fastmail.com Sun Nov 18 11:40:10 2018 From: elliotnunn at fastmail.com (Elliot Nunn) Date: Sun, 18 Nov 2018 11:40:10 +0800 Subject: [CDG5] CubeE status In-Reply-To: <9D81091C-DECC-49D0-A1E1-FA6B9F3C5F16@verizon.net> References: <0BBA2D2D-C537-4A0B-9E75-2B5929F78389@fastmail.com> <6A66F548-8937-43C1-9AD5-7B6EC73131E9@verizon.net> <92A69E21-F132-4677-878D-D7D1D001E876@fastmail.com> <33E06896-6F6F-4E6C-92C5-6088152B04A1@verizon.net> <6DC0C71C-8E55-4406-AF39-B7C34F6DBA62@fastmail.com> <9D81091C-DECC-49D0-A1E1-FA6B9F3C5F16@verizon.net> Message-ID: <154DF078-57E3-48B8-A252-008D40C309F2@fastmail.com> My choice of System version (7.0.1), MPW version (3.2.3) and Interfaces/Libraries could certainly be critiqued. Right now it cannot build anything PowerPC. System 7.0.1 is nice and quick, and has little extra cruft, but OTOH the 7.5.3 in empw supporst the very latest MPW version while still running on 24-bit systems. System 6, I found early on, is out of the question, because many MPW tools came to depend on FSSpec support early in the 7 era. Two questions then: 1. Should BuildCubeE (better name?) allow a choice of multiple software versions? 2. Should you just include your core build tools with your source? > On 18 Nov 2018, at 11:29 am, Daniel B-J wrote: > >> Phew! Thanks for testing > > No problem. > >> If you have sources in a disk image, just run DumpHFS, a command line tool in >> machfs. > > DumpHFS works great. One of my smaller test tools is now on Github: https://github.com/DBJ314/GetPMFeatures. I'll eventually stick my other projects on there as well. > > Can't build it using BuildCubeE, because of the lack of InterfaceLib and other important libraries. Not really a problem as BuildCubeE isn't meant for that yet. > >> If you have newworld-rom working, you can just: >> >> dd if=CubeE/BuildResults/RISC/Image/RomMondo of=newworld-rom/rom conv=notrunc > > Seems to work. Will test the tbxi later. > >> I hope to evolve BuildCubeE into the "general solution" to MPW builds >> (possibly CodeWarrior too?). > > Cool. > >> . Any chance you're keen to fix up the SizeMem >> code that I have mislabelled as "Lastly"? > Not likely. I am still a master procrastinator. > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From danielbj314 at verizon.net Sun Nov 18 11:57:11 2018 From: danielbj314 at verizon.net (Daniel B-J) Date: Sat, 17 Nov 2018 22:57:11 -0500 Subject: [CDG5] CubeE status In-Reply-To: <154DF078-57E3-48B8-A252-008D40C309F2@fastmail.com> References: <0BBA2D2D-C537-4A0B-9E75-2B5929F78389@fastmail.com> <6A66F548-8937-43C1-9AD5-7B6EC73131E9@verizon.net> <92A69E21-F132-4677-878D-D7D1D001E876@fastmail.com> <33E06896-6F6F-4E6C-92C5-6088152B04A1@verizon.net> <6DC0C71C-8E55-4406-AF39-B7C34F6DBA62@fastmail.com> <9D81091C-DECC-49D0-A1E1-FA6B9F3C5F16@verizon.net> <154DF078-57E3-48B8-A252-008D40C309F2@fastmail.com> Message-ID: <282BEAE1-80A4-4572-8CE7-F482EFDC3DEB@verizon.net> > My choice of System version (7.0.1), MPW version (3.2.3) and > Interfaces/Libraries could certainly be critiqued. It builds CubeE, which is all that matters at the moment. > Two questions then: > 1. Should BuildCubeE (better name?) allow a choice of multiple software > versions? > 2. Should you just include your core build tools with your source? I think you should either have 2 defaults (one to build CubeE and one that is the very latest) which can be chosen or overridden completely as a command option or no defaults and have it be with the repo you are building or specified by command line. Not sure which is the best, though the ability to merge together arbitrary images for build purposes would be pretty neat. From elliotnunn at fastmail.com Tue Nov 20 18:25:08 2018 From: elliotnunn at fastmail.com (Elliot Nunn) Date: Tue, 20 Nov 2018 18:25:08 +0800 Subject: [CDG5] Notification Manager Message-ID: Got the Notification Manager (almost all C) round-tripping. Not an especially important manager, but because I tried and failed to round-trip it from source last year, this is very satisfying. From jason.duerstock at gmail.com Wed Nov 21 05:01:32 2018 From: jason.duerstock at gmail.com (Jason Duerstock) Date: Tue, 20 Nov 2018 16:01:32 -0500 Subject: [CDG5] Notification Manager In-Reply-To: References: Message-ID: Rock on! On Tue, Nov 20, 2018 at 2:06 PM Elliot Nunn wrote: > > Got the Notification Manager (almost all C) round-tripping. Not an > especially important manager, but because I tried and failed to > round-trip it from source last year, this is very satisfying. > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From elliotnunn at fastmail.com Wed Nov 21 10:03:48 2018 From: elliotnunn at fastmail.com (Elliot Nunn) Date: Wed, 21 Nov 2018 10:03:48 +0800 Subject: [CDG5] Notification Manager In-Reply-To: References: Message-ID: <0B45A04A-B5C9-4206-945B-D23D5256520F@fastmail.com> :D Welcome back! > On 21 Nov 2018, at 5:01 am, Jason Duerstock wrote: > > Rock on! > On Tue, Nov 20, 2018 at 2:06 PM Elliot Nunn wrote: >> >> Got the Notification Manager (almost all C) round-tripping. Not an >> especially important manager, but because I tried and failed to >> round-trip it from source last year, this is very satisfying. >> _______________________________________________ >> cdg5 mailing list >> cdg5 at ucc.asn.au >> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From jason.duerstock at gmail.com Wed Nov 21 11:54:29 2018 From: jason.duerstock at gmail.com (Jason Duerstock) Date: Tue, 20 Nov 2018 22:54:29 -0500 Subject: [CDG5] Notification Manager In-Reply-To: <0B45A04A-B5C9-4206-945B-D23D5256520F@fastmail.com> References: <0B45A04A-B5C9-4206-945B-D23D5256520F@fastmail.com> Message-ID: Thanks! I know it's been a while, and you and the gang have clearly made some wild progress. Is there a general to-do list (with or without priorities) somewhere? On Tue, Nov 20, 2018 at 9:04 PM Elliot Nunn wrote: > > :D Welcome back! > > > On 21 Nov 2018, at 5:01 am, Jason Duerstock wrote: > > > > Rock on! > > On Tue, Nov 20, 2018 at 2:06 PM Elliot Nunn wrote: > >> > >> Got the Notification Manager (almost all C) round-tripping. Not an > >> especially important manager, but because I tried and failed to > >> round-trip it from source last year, this is very satisfying. > >> _______________________________________________ > >> cdg5 mailing list > >> cdg5 at ucc.asn.au > >> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 > > _______________________________________________ > > cdg5 mailing list > > cdg5 at ucc.asn.au > > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 > > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From elliotnunn at fastmail.com Wed Nov 21 19:18:49 2018 From: elliotnunn at fastmail.com (Elliot Nunn) Date: Wed, 21 Nov 2018 19:18:49 +0800 Subject: [CDG5] Notification Manager In-Reply-To: References: <0B45A04A-B5C9-4206-945B-D23D5256520F@fastmail.com> Message-ID: <1C1F0233-8314-4844-8E13-1ABC6D919154@fastmail.com> No central list, just productive chaos. For my own part, in August last year I took a deep dive into the CubeE source tree and got it round-tripping with some help from a ROM-to-object-file tool. But actual reversed sources only made up 30% of the "main code" binary, and those were all asm. Lately I found the correct version of Apple's C compiler and got some of the C sources round-tripping. This leaves nearly half of the ROM being built from source. Here is the actual source, and my you-beaut build script: https://github.com/elliotnunn/CubeE https://github.com/elliotnunn/BuildCubeE Given your 68k asm skills, it would be awesome to get you on board with the CubeE project. I also have a comprehensive reversal of the whole NanoKernel version history on hold, which I will get to over the summer. Daniel and Max have great stuff going on, but they'd be the best people to show it off. > On 21 Nov 2018, at 11:54 am, Jason Duerstock wrote: > > Thanks! I know it's been a while, and you and the gang have clearly > made some wild progress. Is there a general to-do list (with or > without priorities) somewhere? > On Tue, Nov 20, 2018 at 9:04 PM Elliot Nunn wrote: >> >> :D Welcome back! >> >>> On 21 Nov 2018, at 5:01 am, Jason Duerstock wrote: >>> >>> Rock on! >>> On Tue, Nov 20, 2018 at 2:06 PM Elliot Nunn wrote: >>>> >>>> Got the Notification Manager (almost all C) round-tripping. Not an >>>> especially important manager, but because I tried and failed to >>>> round-trip it from source last year, this is very satisfying. >>>> _______________________________________________ >>>> cdg5 mailing list >>>> cdg5 at ucc.asn.au >>>> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 >>> _______________________________________________ >>> cdg5 mailing list >>> cdg5 at ucc.asn.au >>> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 >> >> _______________________________________________ >> cdg5 mailing list >> cdg5 at ucc.asn.au >> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From jason.duerstock at gmail.com Thu Nov 22 01:15:40 2018 From: jason.duerstock at gmail.com (Jason Duerstock) Date: Wed, 21 Nov 2018 12:15:40 -0500 Subject: [CDG5] Notification Manager In-Reply-To: <1C1F0233-8314-4844-8E13-1ABC6D919154@fastmail.com> References: <0B45A04A-B5C9-4206-945B-D23D5256520F@fastmail.com> <1C1F0233-8314-4844-8E13-1ABC6D919154@fastmail.com> Message-ID: Has anyone figured out to get VM turned on in 9.2 if you have >=1 GB RAM? On Wed, Nov 21, 2018 at 6:19 AM Elliot Nunn wrote: > > No central list, just productive chaos. > > For my own part, in August last year I took a deep dive into the CubeE > source tree and got it round-tripping with some help from a > ROM-to-object-file tool. But actual reversed sources only made up 30% of > the "main code" binary, and those were all asm. Lately I found the > correct version of Apple's C compiler and got some of the C sources > round-tripping. This leaves nearly half of the ROM being built from > source. Here is the actual source, and my you-beaut build script: > > https://github.com/elliotnunn/CubeE > https://github.com/elliotnunn/BuildCubeE > > Given your 68k asm skills, it would be awesome to get you on board with > the CubeE project. I also have a comprehensive reversal of the whole > NanoKernel version history on hold, which I will get to over the summer. > > Daniel and Max have great stuff going on, but they'd be the best people > to show it off. > > > On 21 Nov 2018, at 11:54 am, Jason Duerstock wrote: > > > > Thanks! I know it's been a while, and you and the gang have clearly > > made some wild progress. Is there a general to-do list (with or > > without priorities) somewhere? > > On Tue, Nov 20, 2018 at 9:04 PM Elliot Nunn wrote: > >> > >> :D Welcome back! > >> > >>> On 21 Nov 2018, at 5:01 am, Jason Duerstock wrote: > >>> > >>> Rock on! > >>> On Tue, Nov 20, 2018 at 2:06 PM Elliot Nunn wrote: > >>>> > >>>> Got the Notification Manager (almost all C) round-tripping. Not an > >>>> especially important manager, but because I tried and failed to > >>>> round-trip it from source last year, this is very satisfying. > >>>> _______________________________________________ > >>>> cdg5 mailing list > >>>> cdg5 at ucc.asn.au > >>>> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 > >>> _______________________________________________ > >>> cdg5 mailing list > >>> cdg5 at ucc.asn.au > >>> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 > >> > >> _______________________________________________ > >> cdg5 mailing list > >> cdg5 at ucc.asn.au > >> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 > > _______________________________________________ > > cdg5 mailing list > > cdg5 at ucc.asn.au > > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 > > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From danielbj314 at verizon.net Thu Nov 22 01:29:49 2018 From: danielbj314 at verizon.net (daniel B-J) Date: Wed, 21 Nov 2018 12:29:49 -0500 Subject: [CDG5] Notification Manager In-Reply-To: References: <0B45A04A-B5C9-4206-945B-D23D5256520F@fastmail.com> <1C1F0233-8314-4844-8E13-1ABC6D919154@fastmail.com> Message-ID: > On Nov 21, 2018, at 12:15 PM, Jason Duerstock wrote: > > Has anyone figured out to get VM turned on in 9.2 if you have >=1 GB RAM? I think there is a hardcoded constant in the ?ptch? 420 resource in the system file (?ptch? 42 is 68k vm, ?ptch? 420 is ppc vm). Changing that might work. If we could build that from c, then we could have all sorts of fun :) From jason.duerstock at gmail.com Thu Nov 22 02:50:06 2018 From: jason.duerstock at gmail.com (Jason Duerstock) Date: Wed, 21 Nov 2018 13:50:06 -0500 Subject: [CDG5] Notification Manager In-Reply-To: References: <0B45A04A-B5C9-4206-945B-D23D5256520F@fastmail.com> <1C1F0233-8314-4844-8E13-1ABC6D919154@fastmail.com> Message-ID: Thanks for the pointer. I haven't found the constant yet, unless the limit happens to be 64 MB, but I did find where they put the file mapping API. It's curious that they built it with m68k code rather than ppc though. It would be interesting to see statistics regarding how often a typical system switches in and out of m68k emulation. On Wed, Nov 21, 2018 at 12:30 PM daniel B-J wrote: > > > > On Nov 21, 2018, at 12:15 PM, Jason Duerstock wrote: > > > > Has anyone figured out to get VM turned on in 9.2 if you have >=1 GB RAM? > > I think there is a hardcoded constant in the ?ptch? 420 resource in the system file (?ptch? 42 is 68k vm, ?ptch? 420 is ppc vm). Changing that might work. > > If we could build that from c, then we could have all sorts of fun :) > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From danielbj314 at verizon.net Thu Nov 22 03:33:45 2018 From: danielbj314 at verizon.net (Daniel B-J) Date: Wed, 21 Nov 2018 14:33:45 -0500 Subject: [CDG5] Notification Manager In-Reply-To: References: <0B45A04A-B5C9-4206-945B-D23D5256520F@fastmail.com> <1C1F0233-8314-4844-8E13-1ABC6D919154@fastmail.com> Message-ID: <86EBEB60-EEAD-4FA0-BB3F-365F77A120EF@verizon.net> > Thanks for the pointer. I haven't found the constant yet In the 'ptch' 420 I have, the comparison is a CmpI.L #$3E700000, D3 at offset 0x6D46 from the start of the file. Just searching for 3E700000 will probably do the trick. > It's curious that they built it with m68k code rather > than ppc though. Yeah. For some reason NKv1.x doesn't let the PowerPC context use any of the special calls (except kcReturnFromException). Not sure why. From jason.duerstock at gmail.com Fri Nov 23 00:31:46 2018 From: jason.duerstock at gmail.com (Jason Duerstock) Date: Thu, 22 Nov 2018 11:31:46 -0500 Subject: [CDG5] Notification Manager In-Reply-To: <86EBEB60-EEAD-4FA0-BB3F-365F77A120EF@verizon.net> References: <0B45A04A-B5C9-4206-945B-D23D5256520F@fastmail.com> <1C1F0233-8314-4844-8E13-1ABC6D919154@fastmail.com> <86EBEB60-EEAD-4FA0-BB3F-365F77A120EF@verizon.net> Message-ID: Another question for you all -- does NK v2.x support separate address spaces, a la any sane Unix system? It's not readily apparent to me from the API reading. On Wed, Nov 21, 2018 at 2:34 PM Daniel B-J wrote: > > > Thanks for the pointer. I haven't found the constant yet > > In the 'ptch' 420 I have, the comparison is a CmpI.L #$3E700000, D3 at offset 0x6D46 from the start of the file. Just searching for 3E700000 will probably do the trick. > > > > It's curious that they built it with m68k code rather > > than ppc though. > > Yeah. For some reason NKv1.x doesn't let the PowerPC context use any of the special calls (except kcReturnFromException). Not sure why. > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From maximumspatium at googlemail.com Fri Nov 23 01:18:16 2018 From: maximumspatium at googlemail.com (Max Poliakovski) Date: Thu, 22 Nov 2018 18:18:16 +0100 Subject: [CDG5] Notification Manager In-Reply-To: References: <0B45A04A-B5C9-4206-945B-D23D5256520F@fastmail.com> <1C1F0233-8314-4844-8E13-1ABC6D919154@fastmail.com> <86EBEB60-EEAD-4FA0-BB3F-365F77A120EF@verizon.net> Message-ID: Quoting Ren? Vega, the author of the NK v2.x: I rewrote the nanokernel in OS 8.6, adding multitasking, *multiple address > space support*, fully protected memory, multiprocessing > support, a high performance scheduler, enhanced power management. A support is also there. The question is how to access this feature. A large bulk of NK SPI is still undocumented so I cannot say for sure. As far as I know, OS9 utilizes a small subset of the modern NK SPI; it's practically stuck to NK v1.x. The rest of v2 was designed for a further OS9 evolution (outside of the blue environment) that unfortunately never happened... Am Do., 22. Nov. 2018 um 17:32 Uhr schrieb Jason Duerstock < jason.duerstock at gmail.com>: > Another question for you all -- does NK v2.x support separate address > spaces, a la any sane Unix system? It's not readily apparent to me > from the API reading. > -------------- next part -------------- An HTML attachment was scrubbed... URL: https://lists.ucc.gu.uwa.edu.au/pipermail/cdg5/attachments/20181122/57ef1810/attachment.htm From danielbj314 at verizon.net Fri Nov 23 01:37:25 2018 From: danielbj314 at verizon.net (daniel B-J) Date: Thu, 22 Nov 2018 12:37:25 -0500 Subject: [CDG5] Notification Manager In-Reply-To: References: <0B45A04A-B5C9-4206-945B-D23D5256520F@fastmail.com> <1C1F0233-8314-4844-8E13-1ABC6D919154@fastmail.com> <86EBEB60-EEAD-4FA0-BB3F-365F77A120EF@verizon.net> Message-ID: <3E9B1C47-5F90-43BF-903D-554A69FA19B1@verizon.net> The NK kinda sorta supports multiple address spaces. You can create them, delete them, add Areas to them, and set MPTasks to use them, but they still use the PMDTs as a fallback mechanism. As it currently stands in NKv2.28, a blank address space with no Areas will still map the io memory regions and main memory (which might possibly ignore any changes made by the vm code. Not sure). This can be overridden by placing an Area that overlaps that region, but there is currently no way to turn the feature off. From maximumspatium at googlemail.com Fri Nov 23 01:46:02 2018 From: maximumspatium at googlemail.com (Max Poliakovski) Date: Thu, 22 Nov 2018 18:46:02 +0100 Subject: [CDG5] Notification Manager In-Reply-To: <3E9B1C47-5F90-43BF-903D-554A69FA19B1@verizon.net> References: <0B45A04A-B5C9-4206-945B-D23D5256520F@fastmail.com> <1C1F0233-8314-4844-8E13-1ABC6D919154@fastmail.com> <86EBEB60-EEAD-4FA0-BB3F-365F77A120EF@verizon.net> <3E9B1C47-5F90-43BF-903D-554A69FA19B1@verizon.net> Message-ID: Is it true that external exceptions are always routed via the 68k emulator, even in NK 2.x? Am Do., 22. Nov. 2018 um 18:38 Uhr schrieb daniel B-J < danielbj314 at verizon.net>: > The NK kinda sorta supports multiple address spaces. > > You can create them, delete them, add Areas to them, and set MPTasks to > use them, but they still use the PMDTs as a fallback mechanism. > > As it currently stands in NKv2.28, a blank address space with no Areas > will still map the io memory regions and main memory (which might possibly > ignore any changes made by the vm code. Not sure). > > This can be overridden by placing an Area that overlaps that region, but > there is currently no way to turn the feature off. > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 > -------------- next part -------------- An HTML attachment was scrubbed... URL: https://lists.ucc.gu.uwa.edu.au/pipermail/cdg5/attachments/20181122/4fdf6b45/attachment.htm From danielbj314 at verizon.net Fri Nov 23 02:04:00 2018 From: danielbj314 at verizon.net (daniel B-J) Date: Thu, 22 Nov 2018 13:04:00 -0500 Subject: [CDG5] Notification Manager In-Reply-To: References: <0B45A04A-B5C9-4206-945B-D23D5256520F@fastmail.com> <1C1F0233-8314-4844-8E13-1ABC6D919154@fastmail.com> <86EBEB60-EEAD-4FA0-BB3F-365F77A120EF@verizon.net> <3E9B1C47-5F90-43BF-903D-554A69FA19B1@verizon.net> Message-ID: > Is it true that external exceptions are always routed via the 68k emulator, even in NK 2.x? I think so. Note that even in NKv1.x, you could set the NCB to enable ecNoException. Then, it would trigger the exception handler for PowerPC rather than going to the emulator. But I don?t think Apple ever did that. From jason.duerstock at gmail.com Wed Nov 28 00:57:35 2018 From: jason.duerstock at gmail.com (Jason Duerstock) Date: Tue, 27 Nov 2018 11:57:35 -0500 Subject: [CDG5] Random questions Message-ID: 1) Does anyone know how things get loaded from the "Classic" file in the System Folder? Does this happen under native 9.x, or only under OS X? 2) If you haven't seen these before: https://github.com/SolraBizna/MacLua5.3 https://github.com/fblondiau/ToolDaemon http://www.kallisys.com/files/mac/toolserver(1)-1.2.dmg 3) Has anyone looked into compressed resources and the associate 'dcmp' and 'ncmp' resources? I'm wondering if it would be handy to have an lz4 resourcecompressor/decompressor. Or if there would be another lossless codec that would be more appropriate. Thanks, Jason From elliotnunn at fastmail.com Wed Nov 28 10:59:32 2018 From: elliotnunn at fastmail.com (Elliot Nunn) Date: Wed, 28 Nov 2018 10:59:32 +0800 Subject: [CDG5] Random questions In-Reply-To: References: Message-ID: 1. The Classic file seems to act as a System Enabler ("Gibbly"). 2. Hadn't seen that new Lua port -- very cool! I wonder if MPW could eventually be persuaded to run binary Tools reentrantly. It would obviate a lot of MPW Shell hackery. Sadly I had trouble ever getting ToolDaemon to work. My workaround for a while was to SSH into a Tiger machine and send an Apple Event to ToolServer from there. 3. I have briefly looked into the on-disk encoding of compressed resources, in order to prevent [SimpleDeRez](https://pypi.org/project/macresources/) from choking on them. The scheme seems to rely on executable resources in the same file, and I'm naturally not keen to emulate those. But if you have some docs on the common format(s), then I am all ears! Bin-patching compressed resources for BuildCubeE has been frustrating. So far I have just opened them in ResEdit for decompression. > On 28 Nov 2018, at 12:57 am, Jason Duerstock wrote: > > 1) Does anyone know how things get loaded from the "Classic" file in > the System Folder? Does this happen under native 9.x, or only under > OS X? > > 2) If you haven't seen these before: > https://github.com/SolraBizna/MacLua5.3 > https://github.com/fblondiau/ToolDaemon > http://www.kallisys.com/files/mac/toolserver(1)-1.2.dmg > > 3) Has anyone looked into compressed resources and the associate > 'dcmp' and 'ncmp' resources? I'm wondering if it would be handy to > have an lz4 resourcecompressor/decompressor. Or if there would be > another lossless codec that would be more appropriate. > > Thanks, > > Jason > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From jason.duerstock at gmail.com Wed Nov 28 22:44:48 2018 From: jason.duerstock at gmail.com (Jason Duerstock) Date: Wed, 28 Nov 2018 09:44:48 -0500 Subject: [CDG5] Random questions In-Reply-To: References: Message-ID: I haven't found a whole lot about resource compression so far, but there is this: http://preserve.mactech.com/articles/mactech/Vol.09/09.01/ResCompression/index.html Surprisingly, there is also this tidbit from: http://preserve.mactech.com/articles/mactech/Vol.12/12.08/OS8Checklist/index.html "9. Don?t use compressed resources. Resources which are decompressed with the undocumented ?dcmp? mechanism in System 7 will not be compatible with Mac OS 8. Resources which are compressed with other third-party mechanisms such as Application VISE will continue to work." Considering they're still being used in OS 9, I don't know what to think of that other than "huh?". The following files detail the "Donn" and "Greggy" codecs: https://github.com/elliotnunn/CubeE/blob/master/Patches/GreggyBitsDefProc.a https://github.com/elliotnunn/CubeE/blob/master/Patches/DeCompressDefProc.a I can try to make a Python decompress for the above if you want. Jason On Tue, Nov 27, 2018 at 10:00 PM Elliot Nunn wrote: > > 1. The Classic file seems to act as a System Enabler ("Gibbly"). > > 2. Hadn't seen that new Lua port -- very cool! I wonder if MPW could > eventually be persuaded to run binary Tools reentrantly. It would > obviate a lot of MPW Shell hackery. > > Sadly I had trouble ever getting ToolDaemon to work. My workaround > for a while was to SSH into a Tiger machine and send an Apple Event > to ToolServer from there. > > 3. I have briefly looked into the on-disk encoding of compressed > resources, in order to prevent > [SimpleDeRez](https://pypi.org/project/macresources/) from choking on > them. The scheme seems to rely on executable resources in the same > file, and I'm naturally not keen to emulate those. But if you have > some docs on the common format(s), then I am all ears! > > Bin-patching compressed resources for BuildCubeE has been > frustrating. So far I have just opened them in ResEdit for > decompression. > > > On 28 Nov 2018, at 12:57 am, Jason Duerstock wrote: > > > > 1) Does anyone know how things get loaded from the "Classic" file in > > the System Folder? Does this happen under native 9.x, or only under > > OS X? > > > > 2) If you haven't seen these before: > > https://github.com/SolraBizna/MacLua5.3 > > https://github.com/fblondiau/ToolDaemon > > http://www.kallisys.com/files/mac/toolserver(1)-1.2.dmg > > > > 3) Has anyone looked into compressed resources and the associate > > 'dcmp' and 'ncmp' resources? I'm wondering if it would be handy to > > have an lz4 resourcecompressor/decompressor. Or if there would be > > another lossless codec that would be more appropriate. > > > > Thanks, > > > > Jason > > _______________________________________________ > > cdg5 mailing list > > cdg5 at ucc.asn.au > > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 > > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From danielbj314 at verizon.net Thu Nov 29 06:16:02 2018 From: danielbj314 at verizon.net (Daniel B-J) Date: Wed, 28 Nov 2018 17:16:02 -0500 Subject: [CDG5] Random questions In-Reply-To: References: Message-ID: <78B7B568-EB68-472B-B873-1B74575F93B8@verizon.net> Wow. This looks like such a big security flaw. Compression resources are code that can be run inside a resource file. Resources can be marked 'preload', which makes them run when the file is open. Some files, such as the Desktop Database or any file with a custom icon, can be opened by the Finder automatically under certain circumstances. I'm pretty sure I could make a file that has code within it that gets run as soon as the disk is inserted. As another fun trick, what happens if a 'dcmp' is marked compressed and says it gets decompressed by itself? Endless loop ending in a stack overflow? File of crash? > On Nov 28, 2018, at 9:44 AM, Jason Duerstock wrote: > > I haven't found a whole lot about resource compression so far, but > there is this: > > http://preserve.mactech.com/articles/mactech/Vol.09/09.01/ResCompression/index.html > > Surprisingly, there is also this tidbit from: > http://preserve.mactech.com/articles/mactech/Vol.12/12.08/OS8Checklist/index.html > > "9. Don?t use compressed resources. Resources which are decompressed > with the undocumented ?dcmp? mechanism in System 7 will not be > compatible with Mac OS 8. Resources which are compressed with other > third-party mechanisms such as Application VISE will continue to > work." > > Considering they're still being used in OS 9, I don't know what to > think of that other than "huh?". > > The following files detail the "Donn" and "Greggy" codecs: > > https://github.com/elliotnunn/CubeE/blob/master/Patches/GreggyBitsDefProc.a > https://github.com/elliotnunn/CubeE/blob/master/Patches/DeCompressDefProc.a > > I can try to make a Python decompress for the above if you want. > > Jason > > On Tue, Nov 27, 2018 at 10:00 PM Elliot Nunn wrote: >> >> 1. The Classic file seems to act as a System Enabler ("Gibbly"). >> >> 2. Hadn't seen that new Lua port -- very cool! I wonder if MPW could >> eventually be persuaded to run binary Tools reentrantly. It would >> obviate a lot of MPW Shell hackery. >> >> Sadly I had trouble ever getting ToolDaemon to work. My workaround >> for a while was to SSH into a Tiger machine and send an Apple Event >> to ToolServer from there. >> >> 3. I have briefly looked into the on-disk encoding of compressed >> resources, in order to prevent >> [SimpleDeRez](https://pypi.org/project/macresources/) from choking on >> them. The scheme seems to rely on executable resources in the same >> file, and I'm naturally not keen to emulate those. But if you have >> some docs on the common format(s), then I am all ears! >> >> Bin-patching compressed resources for BuildCubeE has been >> frustrating. So far I have just opened them in ResEdit for >> decompression. >> >>> On 28 Nov 2018, at 12:57 am, Jason Duerstock wrote: >>> >>> 1) Does anyone know how things get loaded from the "Classic" file in >>> the System Folder? Does this happen under native 9.x, or only under >>> OS X? >>> >>> 2) If you haven't seen these before: >>> https://github.com/SolraBizna/MacLua5.3 >>> https://github.com/fblondiau/ToolDaemon >>> http://www.kallisys.com/files/mac/toolserver(1)-1.2.dmg >>> >>> 3) Has anyone looked into compressed resources and the associate >>> 'dcmp' and 'ncmp' resources? I'm wondering if it would be handy to >>> have an lz4 resourcecompressor/decompressor. Or if there would be >>> another lossless codec that would be more appropriate. >>> >>> Thanks, >>> >>> Jason >>> _______________________________________________ >>> cdg5 mailing list >>> cdg5 at ucc.asn.au >>> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 >> >> _______________________________________________ >> cdg5 mailing list >> cdg5 at ucc.asn.au >> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From danielbj314 at verizon.net Thu Nov 29 07:03:57 2018 From: danielbj314 at verizon.net (Daniel B-J) Date: Wed, 28 Nov 2018 18:03:57 -0500 Subject: [CDG5] Random questions In-Reply-To: <78B7B568-EB68-472B-B873-1B74575F93B8@verizon.net> References: <78B7B568-EB68-472B-B873-1B74575F93B8@verizon.net> Message-ID: <1BE81D7B-CD48-49B8-8B00-B6DA9BF78CBB@verizon.net> On another crazy tangent, do Pippin authentication files use the data fork or the resource fork? If they use the resource fork, we can let the blitter.net guy know the authentication check can be easily bypassed. > On Nov 28, 2018, at 5:16 PM, Daniel B-J wrote: > > Wow. This looks like such a big security flaw. > > Compression resources are code that can be run inside a resource file. > > Resources can be marked 'preload', which makes them run when the file is open. > > Some files, such as the Desktop Database or any file with a custom icon, can be opened by the Finder automatically under certain circumstances. > > I'm pretty sure I could make a file that has code within it that gets run as soon as the disk is inserted. > > As another fun trick, what happens if a 'dcmp' is marked compressed and says it gets decompressed by itself? Endless loop ending in a stack overflow? File of crash? >> On Nov 28, 2018, at 9:44 AM, Jason Duerstock wrote: >> >> I haven't found a whole lot about resource compression so far, but >> there is this: >> >> http://preserve.mactech.com/articles/mactech/Vol.09/09.01/ResCompression/index.html >> >> Surprisingly, there is also this tidbit from: >> http://preserve.mactech.com/articles/mactech/Vol.12/12.08/OS8Checklist/index.html >> >> "9. Don?t use compressed resources. Resources which are decompressed >> with the undocumented ?dcmp? mechanism in System 7 will not be >> compatible with Mac OS 8. Resources which are compressed with other >> third-party mechanisms such as Application VISE will continue to >> work." >> >> Considering they're still being used in OS 9, I don't know what to >> think of that other than "huh?". >> >> The following files detail the "Donn" and "Greggy" codecs: >> >> https://github.com/elliotnunn/CubeE/blob/master/Patches/GreggyBitsDefProc.a >> https://github.com/elliotnunn/CubeE/blob/master/Patches/DeCompressDefProc.a >> >> I can try to make a Python decompress for the above if you want. >> >> Jason >> >> On Tue, Nov 27, 2018 at 10:00 PM Elliot Nunn wrote: >>> >>> 1. The Classic file seems to act as a System Enabler ("Gibbly"). >>> >>> 2. Hadn't seen that new Lua port -- very cool! I wonder if MPW could >>> eventually be persuaded to run binary Tools reentrantly. It would >>> obviate a lot of MPW Shell hackery. >>> >>> Sadly I had trouble ever getting ToolDaemon to work. My workaround >>> for a while was to SSH into a Tiger machine and send an Apple Event >>> to ToolServer from there. >>> >>> 3. I have briefly looked into the on-disk encoding of compressed >>> resources, in order to prevent >>> [SimpleDeRez](https://pypi.org/project/macresources/) from choking on >>> them. The scheme seems to rely on executable resources in the same >>> file, and I'm naturally not keen to emulate those. But if you have >>> some docs on the common format(s), then I am all ears! >>> >>> Bin-patching compressed resources for BuildCubeE has been >>> frustrating. So far I have just opened them in ResEdit for >>> decompression. >>> >>>> On 28 Nov 2018, at 12:57 am, Jason Duerstock wrote: >>>> >>>> 1) Does anyone know how things get loaded from the "Classic" file in >>>> the System Folder? Does this happen under native 9.x, or only under >>>> OS X? >>>> >>>> 2) If you haven't seen these before: >>>> https://github.com/SolraBizna/MacLua5.3 >>>> https://github.com/fblondiau/ToolDaemon >>>> http://www.kallisys.com/files/mac/toolserver(1)-1.2.dmg >>>> >>>> 3) Has anyone looked into compressed resources and the associate >>>> 'dcmp' and 'ncmp' resources? I'm wondering if it would be handy to >>>> have an lz4 resourcecompressor/decompressor. Or if there would be >>>> another lossless codec that would be more appropriate. >>>> >>>> Thanks, >>>> >>>> Jason >>>> _______________________________________________ >>>> cdg5 mailing list >>>> cdg5 at ucc.asn.au >>>> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 >>> >>> _______________________________________________ >>> cdg5 mailing list >>> cdg5 at ucc.asn.au >>> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 >> _______________________________________________ >> cdg5 mailing list >> cdg5 at ucc.asn.au >> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 > > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 -------------- next part -------------- An HTML attachment was scrubbed... URL: https://lists.ucc.gu.uwa.edu.au/pipermail/cdg5/attachments/20181128/44771fbd/attachment.htm From elliotnunn at fastmail.com Thu Nov 29 08:00:12 2018 From: elliotnunn at fastmail.com (Elliot Nunn) Date: Thu, 29 Nov 2018 08:00:12 +0800 Subject: [CDG5] Random questions In-Reply-To: References: Message-ID: <1F162B0E-D1B2-4796-8774-2D79C8414A88@fastmail.com> If you see any documentation promising big changes in "Mac OS 8", think Copland. Of course that was cancelled, and some of its high-level improvements got backported to a System 7-based Mac OS 8 and 9. I note that 'dcmp' 0, 1 and 2 did not change from 7 to 9, but that a 'dcmp' 3 was also added. (None of these resources contains any PowerPC code.) Yes, please implement these in Python! I will get them into macresources. Re-compression would be even better! > On 28 Nov 2018, at 10:44 pm, Jason Duerstock wrote: > > I haven't found a whole lot about resource compression so far, but > there is this: > > http://preserve.mactech.com/articles/mactech/Vol.09/09.01/ResCompression/index.html > > Surprisingly, there is also this tidbit from: > http://preserve.mactech.com/articles/mactech/Vol.12/12.08/OS8Checklist/index.html > > "9. Don?t use compressed resources. Resources which are decompressed > with the undocumented ?dcmp? mechanism in System 7 will not be > compatible with Mac OS 8. Resources which are compressed with other > third-party mechanisms such as Application VISE will continue to > work." > > Considering they're still being used in OS 9, I don't know what to > think of that other than "huh?". > > The following files detail the "Donn" and "Greggy" codecs: > > https://github.com/elliotnunn/CubeE/blob/master/Patches/GreggyBitsDefProc.a > https://github.com/elliotnunn/CubeE/blob/master/Patches/DeCompressDefProc.a > > I can try to make a Python decompress for the above if you want. > > Jason > > On Tue, Nov 27, 2018 at 10:00 PM Elliot Nunn wrote: >> >> 1. The Classic file seems to act as a System Enabler ("Gibbly"). >> >> 2. Hadn't seen that new Lua port -- very cool! I wonder if MPW could >> eventually be persuaded to run binary Tools reentrantly. It would >> obviate a lot of MPW Shell hackery. >> >> Sadly I had trouble ever getting ToolDaemon to work. My workaround >> for a while was to SSH into a Tiger machine and send an Apple Event >> to ToolServer from there. >> >> 3. I have briefly looked into the on-disk encoding of compressed >> resources, in order to prevent >> [SimpleDeRez](https://pypi.org/project/macresources/) from choking on >> them. The scheme seems to rely on executable resources in the same >> file, and I'm naturally not keen to emulate those. But if you have >> some docs on the common format(s), then I am all ears! >> >> Bin-patching compressed resources for BuildCubeE has been >> frustrating. So far I have just opened them in ResEdit for >> decompression. >> >>> On 28 Nov 2018, at 12:57 am, Jason Duerstock wrote: >>> >>> 1) Does anyone know how things get loaded from the "Classic" file in >>> the System Folder? Does this happen under native 9.x, or only under >>> OS X? >>> >>> 2) If you haven't seen these before: >>> https://github.com/SolraBizna/MacLua5.3 >>> https://github.com/fblondiau/ToolDaemon >>> http://www.kallisys.com/files/mac/toolserver(1)-1.2.dmg >>> >>> 3) Has anyone looked into compressed resources and the associate >>> 'dcmp' and 'ncmp' resources? I'm wondering if it would be handy to >>> have an lz4 resourcecompressor/decompressor. Or if there would be >>> another lossless codec that would be more appropriate. >>> >>> Thanks, >>> >>> Jason >>> _______________________________________________ >>> cdg5 mailing list >>> cdg5 at ucc.asn.au >>> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 >> >> _______________________________________________ >> cdg5 mailing list >> cdg5 at ucc.asn.au >> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From elliotnunn at fastmail.com Thu Nov 29 15:58:12 2018 From: elliotnunn at fastmail.com (Elliot Nunn) Date: Thu, 29 Nov 2018 15:58:12 +0800 Subject: [CDG5] Update on Thing Manager Message-ID: <1634A494-142B-4526-B20D-E4FF502031B4@fastmail.com> Did some work on the Thing ("Component") Manager today. Very frustrating. Long C functions are super hard to diff. Will come back to it. From elliotnunn at fastmail.com Thu Nov 29 18:02:35 2018 From: elliotnunn at fastmail.com (Elliot Nunn) Date: Thu, 29 Nov 2018 18:02:35 +0800 Subject: [CDG5] Daniel: SizeMem Message-ID: Thanks, Daniel, for letting me know a few months ago that the code I had dumped as "Lastly.lib" was actually from SizeMem.a. As part of my effort to rip the "amphibian DNA" out of the Cube-E repo, I have fixed this. The result helps our understanding of VMAllocateMemory, the DR Emulator, and the Emulator Control Registers: https://github.com/elliotnunn/CubeE/blob/master/OS/StartMgr/SizeMem.a?ts=4#L433 From danielbj314 at verizon.net Thu Nov 29 20:15:38 2018 From: danielbj314 at verizon.net (Daniel B-J) Date: Thu, 29 Nov 2018 07:15:38 -0500 Subject: [CDG5] Daniel: SizeMem In-Reply-To: References: Message-ID: <148E5EC5-9532-4939-B71D-2F8D201D1883@verizon.net> Wow, yeah. That code is connected to so many other pieces. I think my reasons for looking at it were "I think this reads InfoRecords, what does it do?". > On Nov 29, 2018, at 5:02 AM, Elliot Nunn wrote: > > Thanks, Daniel, for letting me know a few months ago that the code I had > dumped as "Lastly.lib" was actually from SizeMem.a. As part of my effort > to rip the "amphibian DNA" out of the Cube-E repo, I have fixed this. > The result helps our understanding of VMAllocateMemory, the DR Emulator, > and the Emulator Control Registers: > > https://github.com/elliotnunn/CubeE/blob/master/OS/StartMgr/SizeMem.a?ts=4#L433 > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From jason.duerstock at gmail.com Fri Nov 30 02:17:43 2018 From: jason.duerstock at gmail.com (Jason Duerstock) Date: Thu, 29 Nov 2018 13:17:43 -0500 Subject: [CDG5] Random questions In-Reply-To: <1F162B0E-D1B2-4796-8774-2D79C8414A88@fastmail.com> References: <1F162B0E-D1B2-4796-8774-2D79C8414A88@fastmail.com> Message-ID: I believe the 'dcmp' resources were reimplemented as 'ncmp' for PPC code. I will try to take a stab at the Donn and Greggy stuff in the next week or so. Also, if you haven't seen it: https://github.com/avast-tl/retdec It's not really any use on the m68k code, but it does work against PPC. You have to trim out the raw bits from the PEF file you want to process though. Jason On Wed, Nov 28, 2018 at 7:01 PM Elliot Nunn wrote: > > If you see any documentation promising big changes in "Mac OS 8", think > Copland. Of course that was cancelled, and some of its high-level > improvements got backported to a System 7-based Mac OS 8 and 9. > > I note that 'dcmp' 0, 1 and 2 did not change from 7 to 9, but that a > 'dcmp' 3 was also added. (None of these resources contains any PowerPC > code.) Yes, please implement these in Python! I will get them into > macresources. Re-compression would be even better! > > > On 28 Nov 2018, at 10:44 pm, Jason Duerstock wrote: > > > > I haven't found a whole lot about resource compression so far, but > > there is this: > > > > http://preserve.mactech.com/articles/mactech/Vol.09/09.01/ResCompression/index.html > > > > Surprisingly, there is also this tidbit from: > > http://preserve.mactech.com/articles/mactech/Vol.12/12.08/OS8Checklist/index.html > > > > "9. Don?t use compressed resources. Resources which are decompressed > > with the undocumented ?dcmp? mechanism in System 7 will not be > > compatible with Mac OS 8. Resources which are compressed with other > > third-party mechanisms such as Application VISE will continue to > > work." > > > > Considering they're still being used in OS 9, I don't know what to > > think of that other than "huh?". > > > > The following files detail the "Donn" and "Greggy" codecs: > > > > https://github.com/elliotnunn/CubeE/blob/master/Patches/GreggyBitsDefProc.a > > https://github.com/elliotnunn/CubeE/blob/master/Patches/DeCompressDefProc.a > > > > I can try to make a Python decompress for the above if you want. > > > > Jason > > > > On Tue, Nov 27, 2018 at 10:00 PM Elliot Nunn wrote: > >> > >> 1. The Classic file seems to act as a System Enabler ("Gibbly"). > >> > >> 2. Hadn't seen that new Lua port -- very cool! I wonder if MPW could > >> eventually be persuaded to run binary Tools reentrantly. It would > >> obviate a lot of MPW Shell hackery. > >> > >> Sadly I had trouble ever getting ToolDaemon to work. My workaround > >> for a while was to SSH into a Tiger machine and send an Apple Event > >> to ToolServer from there. > >> > >> 3. I have briefly looked into the on-disk encoding of compressed > >> resources, in order to prevent > >> [SimpleDeRez](https://pypi.org/project/macresources/) from choking on > >> them. The scheme seems to rely on executable resources in the same > >> file, and I'm naturally not keen to emulate those. But if you have > >> some docs on the common format(s), then I am all ears! > >> > >> Bin-patching compressed resources for BuildCubeE has been > >> frustrating. So far I have just opened them in ResEdit for > >> decompression. > >> > >>> On 28 Nov 2018, at 12:57 am, Jason Duerstock wrote: > >>> > >>> 1) Does anyone know how things get loaded from the "Classic" file in > >>> the System Folder? Does this happen under native 9.x, or only under > >>> OS X? > >>> > >>> 2) If you haven't seen these before: > >>> https://github.com/SolraBizna/MacLua5.3 > >>> https://github.com/fblondiau/ToolDaemon > >>> http://www.kallisys.com/files/mac/toolserver(1)-1.2.dmg > >>> > >>> 3) Has anyone looked into compressed resources and the associate > >>> 'dcmp' and 'ncmp' resources? I'm wondering if it would be handy to > >>> have an lz4 resourcecompressor/decompressor. Or if there would be > >>> another lossless codec that would be more appropriate. > >>> > >>> Thanks, > >>> > >>> Jason > >>> _______________________________________________ > >>> cdg5 mailing list > >>> cdg5 at ucc.asn.au > >>> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 > >> > >> _______________________________________________ > >> cdg5 mailing list > >> cdg5 at ucc.asn.au > >> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 > > _______________________________________________ > > cdg5 mailing list > > cdg5 at ucc.asn.au > > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 > > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From elliotnunn at fastmail.com Fri Nov 30 13:22:09 2018 From: elliotnunn at fastmail.com (Elliot Nunn) Date: Fri, 30 Nov 2018 13:22:09 +0800 Subject: [CDG5] Random questions In-Reply-To: References: <1F162B0E-D1B2-4796-8774-2D79C8414A88@fastmail.com> Message-ID: <3CF74750-270E-4B95-A199-BB888CD80C0A@fastmail.com> Oops -- right you are! Re RetDec, Max, any thoughts? We could really use a high-throughput PEF-hacking workflow. > On 30 Nov 2018, at 2:17 am, Jason Duerstock wrote: > > I believe the 'dcmp' resources were reimplemented as 'ncmp' for PPC code. > > I will try to take a stab at the Donn and Greggy stuff in the next week or so. > > Also, if you haven't seen it: https://github.com/avast-tl/retdec > > It's not really any use on the m68k code, but it does work against > PPC. You have to trim out the raw bits from the PEF file you want to > process though. > > Jason > On Wed, Nov 28, 2018 at 7:01 PM Elliot Nunn wrote: >> >> If you see any documentation promising big changes in "Mac OS 8", think >> Copland. Of course that was cancelled, and some of its high-level >> improvements got backported to a System 7-based Mac OS 8 and 9. >> >> I note that 'dcmp' 0, 1 and 2 did not change from 7 to 9, but that a >> 'dcmp' 3 was also added. (None of these resources contains any PowerPC >> code.) Yes, please implement these in Python! I will get them into >> macresources. Re-compression would be even better! >> >>> On 28 Nov 2018, at 10:44 pm, Jason Duerstock wrote: >>> >>> I haven't found a whole lot about resource compression so far, but >>> there is this: >>> >>> http://preserve.mactech.com/articles/mactech/Vol.09/09.01/ResCompression/index.html >>> >>> Surprisingly, there is also this tidbit from: >>> http://preserve.mactech.com/articles/mactech/Vol.12/12.08/OS8Checklist/index.html >>> >>> "9. Don?t use compressed resources. Resources which are decompressed >>> with the undocumented ?dcmp? mechanism in System 7 will not be >>> compatible with Mac OS 8. Resources which are compressed with other >>> third-party mechanisms such as Application VISE will continue to >>> work." >>> >>> Considering they're still being used in OS 9, I don't know what to >>> think of that other than "huh?". >>> >>> The following files detail the "Donn" and "Greggy" codecs: >>> >>> https://github.com/elliotnunn/CubeE/blob/master/Patches/GreggyBitsDefProc.a >>> https://github.com/elliotnunn/CubeE/blob/master/Patches/DeCompressDefProc.a >>> >>> I can try to make a Python decompress for the above if you want. >>> >>> Jason >>> >>> On Tue, Nov 27, 2018 at 10:00 PM Elliot Nunn wrote: >>>> >>>> 1. The Classic file seems to act as a System Enabler ("Gibbly"). >>>> >>>> 2. Hadn't seen that new Lua port -- very cool! I wonder if MPW could >>>> eventually be persuaded to run binary Tools reentrantly. It would >>>> obviate a lot of MPW Shell hackery. >>>> >>>> Sadly I had trouble ever getting ToolDaemon to work. My workaround >>>> for a while was to SSH into a Tiger machine and send an Apple Event >>>> to ToolServer from there. >>>> >>>> 3. I have briefly looked into the on-disk encoding of compressed >>>> resources, in order to prevent >>>> [SimpleDeRez](https://pypi.org/project/macresources/) from choking on >>>> them. The scheme seems to rely on executable resources in the same >>>> file, and I'm naturally not keen to emulate those. But if you have >>>> some docs on the common format(s), then I am all ears! >>>> >>>> Bin-patching compressed resources for BuildCubeE has been >>>> frustrating. So far I have just opened them in ResEdit for >>>> decompression. >>>> >>>>> On 28 Nov 2018, at 12:57 am, Jason Duerstock wrote: >>>>> >>>>> 1) Does anyone know how things get loaded from the "Classic" file in >>>>> the System Folder? Does this happen under native 9.x, or only under >>>>> OS X? >>>>> >>>>> 2) If you haven't seen these before: >>>>> https://github.com/SolraBizna/MacLua5.3 >>>>> https://github.com/fblondiau/ToolDaemon >>>>> http://www.kallisys.com/files/mac/toolserver(1)-1.2.dmg >>>>> >>>>> 3) Has anyone looked into compressed resources and the associate >>>>> 'dcmp' and 'ncmp' resources? I'm wondering if it would be handy to >>>>> have an lz4 resourcecompressor/decompressor. Or if there would be >>>>> another lossless codec that would be more appropriate. >>>>> >>>>> Thanks, >>>>> >>>>> Jason >>>>> _______________________________________________ >>>>> cdg5 mailing list >>>>> cdg5 at ucc.asn.au >>>>> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 >>>> >>>> _______________________________________________ >>>> cdg5 mailing list >>>> cdg5 at ucc.asn.au >>>> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 >>> _______________________________________________ >>> cdg5 mailing list >>> cdg5 at ucc.asn.au >>> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 >> >> _______________________________________________ >> cdg5 mailing list >> cdg5 at ucc.asn.au >> https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 > _______________________________________________ > cdg5 mailing list > cdg5 at ucc.asn.au > https://lists.ucc.gu.uwa.edu.au/mailman/listinfo/cdg5 From maximumspatium at googlemail.com Fri Nov 30 20:43:48 2018 From: maximumspatium at googlemail.com (Max Poliakovski) Date: Fri, 30 Nov 2018 13:43:48 +0100 Subject: [CDG5] Random questions In-Reply-To: <3CF74750-270E-4B95-A199-BB888CD80C0A@fastmail.com> References: <1F162B0E-D1B2-4796-8774-2D79C8414A88@fastmail.com> <3CF74750-270E-4B95-A199-BB888CD80C0A@fastmail.com> Message-ID: > Re RetDec, Max, any thoughts? We could really use a high-throughput > PEF-hacking workflow. > Retdec doesn't understand PEF. Teaching it to do that requires presumably a lot of effort. Although I'm not familiar with Retdec internals, I'm afraid it's all about deaingl with low-level stuff like writing parsers in C++ and alike. I don't see what code can be quickly reused with Retdec. On the other side, I recently hacked some bits of PEF parsing into my Recomposer tool. Now we can convert PPC code out of PEF to PseudoC and feed it into ScratchABlock that has the unique property to be largely ABI and ISA independent. We can even decompile 68k code with it as long as we can map it to PseudoC. All I need is to rip PEF containers out of resource forks and write them into files. You should already have a tool for that, don't you? Feel free to send me that 'ncmp' stuff. I'll see if I can decompile it... Cheers Max -------------- next part -------------- An HTML attachment was scrubbed... URL: https://lists.ucc.gu.uwa.edu.au/pipermail/cdg5/attachments/20181130/916155b4/attachment.htm