An 'early' morning thought about the future of PWL

[Adi Linden]

Hi,

> Sounds like your on to something here...except for building the pwlconfig
> packages (either tar or RPMs) I think most of this is done with pwlconfig
> now...maybe I missed something...

Yup, just use rpm instead of tar...

> RH RPM 6.x--> manual process --> PWL-RPM.patch file --> PWL-RPM
> 
> then
> 
> RH RPM 6.x --> RPM process with PWL-RPM.patch file -> PWL-RPM
> 
> the "PWL-RPM.patch_file" plus the RH source disk is all you need...
> sounds good.

Right... So the initial build of a package is manual, then create a patch
based on the changes made to the original rpm.

pwlconfig would apply the patch to an original source rpm to create a pwl
source rpm...

Then pwlconfig would run rpm with compile options to create a pwl rpm (no
longer a source but a binary rpm)

The pwl extracts the selected binaries (or the whole rpm) into the build
filesystem... 

> > Use a defined mount point for the PeeWeeLinux filesystem being build
> > independed of where PeeWeeLinux has been extracted to. I.e. /mnt/PWL.
> > 
> Hmmm...shouldn't keep this within the development environment ??
> ...maybe Embedded-Build/projects/xxx/mnt...why would you use the same
> mount point for a project specific filesystem  ??

Don't know why I thought that. Too early in the morning, not enough coffee
yet when I came up with it :)

> Ohhh...wait...you want to basically extract all the tar files we have now
> into one pwl file system, then let the user pick from this file system???
> Seems like an added step, not sure of any advantages...
> Ok, I'm confussed...

Nope... The user would pick the files extracted from the rpm so the
filesystem would only contain the selected binaries.

> > Use rpm in a similar mode as the RedHat install disks do. That is install
> > the PWL specific rpms in /mnt/PWL rather than the current file system.
> 
> Ok, not sure what RH does with the RPMS..I thought they just got extracted
> into the /usr/src/linux/ directory structure..again, I think I'd prefer to keep
> everything within the "Embedded-Build" structure, however, any good reason
> not to would sway me :-)

rpm usually uses an absolute path. We need to extract the binary rpm into
/home/adi/Embedded_Build/projects/mnt as a root directory for example...

> ...this creates the PWL-RPM.patch files for each rpm..ok..I envision these
> fitting into the project as follows:
> 
> Embedded-Build/source/patches

We wouldn't need sources any longer as the RedHat source rpms are our
sources... For things that RedHat doesn't provide we would need to create
our own source rpms.

> Scripts or patches or whatever, that are used to turn a lowly source file
> (RH RPM or whatever) into a PWL-RPM file.  After processing these are
> stored in the packages directory:
> 
> Embedded-Build/packages
> Embedded-Build/packages/package A
> Embedded-Build/packages/package B     
> 
> These are the "global" optimized binaries used for every
> project...these can be tar or RPM files, shouldn't be hard to switch...

The use of rpm would make dependencies easier to follow... I think rpm
uses some predetermined build environment (directory structure) under
/usr/src/

> This is where the RPM files reside after being "processed" by the PWL-RPM.patch
> files. The pwlconfig menu item (yet to be added :-) "Build PWL-RPM" would
> let you pick a patch file and generate the PWL-RPM file.  This would
> be inserted into the packages directory.

After the source rpm has been patched it still needs to be compiled by
rpm...

So we have to deal with the following files in the order of creation:

RedHat source rpm  +  PWL patch  =  PWL source rpm
PWL source rpm processed by rpm  =  PWL binary rpm

PWL binary rpm is similar to the existing package.tar

> > 6. Condition the /mnt/PWL filesystem for the new system
> 
> not sure what this is..

Deleting old files from a previous build.

> > 7. Install the selected files from PWL rpms in /mnt/PWL
> > 8. Install any custom config files into /mnt/PWL, overwriting any existing
> > configuration
> 
> I think these should be project specific...I could be building a router and
> a print server and a thin client..these are all in seperate directories.
> 
> I think this is the process I'm doing now with the tar files, only each project
> 
> keeps their file system seperate within the project directory:
> 
> Embedded-Build/projects/router_486/mnt                    
> Embedded-Build/projects/thin_client/mnt                    
> 
> This is a "mount point" where the project (ie. "router_486") gets built from
> the project specific configuration files. All files are extracted and allows
> the user to tweak the project before it is loaded on to some device. at this
> point, we are device independent.

Right!

> Embedded-Build/mnt
> 
> When you load a device, it gets mounted in the "Embedded-Build/mnt" directory
> until you unmount it and go to another project.  This mount directory is
> bascially a temporary mount point used for every project, but can be used
> to mount any device.

Right!

> > 8. Offer choice of what to do with the newly created system, i.e. build a
> > ramdisk, load a DOC, make a bootable CD...
> > 
> Almost got this working...I have a new script that does DOC and flash disk...
> Adding the ramdisk and bootable CD should be easy, once I know what needs
> to be done.

Yup... I still have to figure out the loopback device.

> > The advantages I can see:
> > - The PWL distribution would no longer require any sources but only patch
> > files containing package description, help information (things that could
> > be displayed by pwlconfig) and the actual patch to the RH source rpm.
> 
> Yes...I have the source, I shouldn't have to download it again...

Actually the sources are only needed to build the PWL binary rpms once.
After that the sources are no longer needed. In fact PWL could still be
distributed in a binary realease with only binary rpm since they don't
change between projects.

> > - Perhaps eases the migration to newer RH versions as they are released.
> > 
> > The disadvantages:
> > - Building an initial PWL development environment could be 'very' time
> > consuming and diskspace consuming since everything has to be compiled
> > first.
> > - Defenitely RH specific!
> > 
> 
> Only RH specific where the patches to the RPMS are concerned. After we
> have built our RPMS (or tar files), the configuration tool does not care 
> what order the files are in or what the directory structure is....

The patches would be RedHat release specific. A RedHat environment would
be needed to build the PWL binary rpms from RedHat source rpms.

The PWL binary release would be distribution independent.

> I'm not sure how this differs from what you have done to build the
> existing tar files.  Now, (or planned real soon now) we do the RPM extract to
> our "Embedded-Build/source" file system, then apply the patches and build tar
> files ...But instead of building tar files, create new RPM files...

Well, the rpms would be extracted to wherever rpms are extracted. Apply
the patch and compile using rpm to build a PWL binary rpm. Coming to think
of it, we would have a source distribution that doesn't have any sources
:). That is, the Redhat RPMS would have to be provided by the user. 

Hmmm... Perhaps only a single menu item would be required in pwlconfig:

Create PWL binary RPMS from Redhat source RPMS

On second thought, some config options like where the RedHat sources are
wouls also be necessary.

> Is there an advantage to building the RPM as opposed to the tar files? maybe
> processing them might be easier...

rpm files have provisins for showing file information and file listing of
files being installed by an 'rpm -ivh <filename>'. I am not that familiar
with rpm other than how to install, upgrade and uninstall binary rpms.

But rpm also has build functions that compile a source rpm to a binary rpm
based on the spec file.

> Shouldn't matter if these are RPMs or TARS, but processing them into specific
> projects should be able to use the same pwlconfig as now, just change from
> tar file processing to RPM file processing ???

That's correct.

> Either way, each project is unique, and built from some "pwl" source
> repository,I guess I need to get up to speed on the RPM stuff too.

Build from the PWL binary rpms. The PWL source rpms are an immediate step.
I don't see any reason for keeping them around. They'd just waste disk
space...

> On a seperate note, I noticed that Transmeta is desiging their Linux to
> work for "Mobile" applications.  OK, that to me means embedded (what else is
> low power for :-)

Yup... 

> Anyhow, while they haven't released anything yet, they have "hinted" that
> they have spent time doing special programs like "touch screen keyboard"
> and "handwritting recognition" (ie.palm pilot work alike programs).
> 
> Also, these are supposedly built from the Debian distribution.
> 
> Soooooo....Assuming that Transmetta has not built any sort of development 
> environment (which true UNIX people *NEVER* do), I think it would be nice
> to be able to get these programs and add them to our embedded RH environment.
> 
> Thus, get their RPM (or whatever Debian uses) and process them into 
> our tar (or RPM) format.  Now you can add these to your project and they
> should just work....

Except Redhat and debian use different patches. So it's not possible to
(in most cases) to use a Debian rpm in a Redhat system (or the other way
around). That said... we manually patch the source rpm prior to building a
PWL source rpm and from that the PWL binary rpm... That patching job would
certainly fix any incompatibilities.

> pwlconfig should allow you to work with applications from any distribution.
> There is nothing specific in "pwlconfig" that would prevent this from 
> working.
> 
> The PeeWee Distribution is useable when we can :
> 
> 1. Create several default projects 
>    (ie. Embedded-Build/projects/[router|thinclient|server|print server|...])
> 
> 2. Create project configurations for each of these projects 
>    (ie. use pwlconfig V1.0 for each of the default projects listed above)

This is distribution independent.

> 3. Process "source files" into optimized binaries 
>    (ie.fill up the Embedded-Build/patches with the PWL-patches_for_RPM_x)

This will most like require a specific Redhat distribution to work
properly. 

A PeeWeeLinux binary distribution would be distribution independent. In
fact, it is now. I build a PeeWeeLinux system on a Redhat 5.2 development
platform.

> On going work is to:
> 
> A. refine pwlconfig for more devices and features
> B. keep building patch files for each new version of RH
> C. build patch files for special applications
> 
> If we can do this, it will be a very usable (and unique) environment for
> embedded Linux development ( and World Domination will be at hand :-)) !!!

:) I see the IPO... opening March 19, 2002 at $20 a share closing on March
20, 2002 at $200 a share... Doing the old RedHat and VALinux stunt... :)

> Hmmm...Let me know if we're on the same page here...I could be missing
> something

Yup... I think we have the same ideas. Like I said at the very beginning
this is dreaming today. For now I think it's important to get what's there
running solid. And I think we'll stick with the manual tar file build dor
now.

However, I found out the hard way (with the router project) that it's
necessary to stay at the 'cutting edge' of development. As USB developes,
etc it is essential to support the 2.4.x kernels, etc. Once a complete
overhaul (recompile) of the PeeWeeLinux binaries is necessary it would be
wise to implement a more streamlined approach. Soooo, this is where my
dreaming comes in. Once that times arrives I want to have made a decision
and a firm plan on how to proceed. Do the brainstorming while my brain is
in creative storming mode. The day will come again soon when my mind is in
the take instructions-do-it mode. No creativity then. 

TTYL,
Adi