ArcBundle format specification
ArcBundles are a method of storing programs in a generic, easily interpretable, archivable, recompressible format that can supply all the necessary functionality for every computer type concievable. This specification is still being worked on, and new features are being added to better accomodate a diverse collection of different computing devices. Many of the ArcBundle features are intended for only one system type, and do not make sense outside of that platform. These features are considered extensions to the base standard, and do not function on all system types. Please make note of which features are supported for your system type when bundling software to prevent unexpected behavior.
ArcBundles are automatically run with emu-run when a user double-clicks on one. This wrapper takes care of unpacking and setting up the program contained. Some of this setup may require additional content to be downloaded from the internet. To enable users to run this software without an internet connection, the emu-install program is also provided. This can be selected by right-clicking on an ArcBundle and selecting install. This second script does all of the tasks involved in setting up a bundle, but doesn't actually run it, allowing it to be used later offline.
At its most basic, an ArcBundle is simply an archive file
containing everything a program needs to run. Nearly every archive
format is supported, including but not limited to ZIP, RAR, and 7z.
7zip archives are preferred, as these archives have fairly good
compression, are easy to create and unpack with many different
programs, and store a file table that can be read without parsing the
entire archive file the way a TAR archive does. This makes them much
easier to work with over a network or internet connection. What
information is stored in the archive is dependent on the original host
- Disk-based console games are stored as BIN/CUE/FLAC format. In the future, an ecm-ripping format will be supported to shrink the bin partition as much as possible. ISO images are not officially supported, and every effort must be taken to avoid these image types when possible. ISO lacks support for some of the odd format quirks that some old computers used, so it can cause strange bugs. For a similar reason, GD-ROM disks (Dreamcast, Naomi) should NOT use BIN, as GD-ROM disks do not fit neatly in this format. The DiskJuggler image format is preferred unless the disk image is from a recompressed CD-G hack. If in doubt, leave Dreamcast images alone.
- Android programs can store resources in their APK directly, but some programs break these resources out into an OBB file. This file's relative location is hard-coded into the APK, but determining the location automatically is very difficult. As a result, Android APKs should be stored bare in an arcbundle, unless they require an OBB. OBB files should be stored in their necessary directory structure inside the OBB subfolder in the arcbundle. Treat the OBB folder as though it were the Android folder on your SD card, and do not include an Android folder inside it. These files will automatically be transferred to the storage drive with the most free space on installation.
- Computer systems with file storage have programs made up of multiple files, so the contents of the directory the program resides in should be archived, including any necessary subfolders. The archive should show the contents of the directory, but not the directory itself. This means that if you've got a single folder stored in the root folder of the archive, you've done it wrong.
- Computer systems that use odd floppy formats, tapes, or
cartridges can simply have a raw dump of these storage formats as a
single file in the archive, provided that it is in a format that
the emulator for that platform can read. For most systems, the
emulator in question is RetroArch, but each RetroArch plugin has its
- Systems with these formats can support multiple versions of a program in a single archive, and the wrappers will automatically display a menu for the user to select which version to run. This really cuts down on the size of a set of files when the archive is created as a solid archive, because the archive then effectively only stores the changes from one file for all the rest.
Other similar formats exist, and some overlap the functionality here. A few are outlined below, with info on support through these wrappers.
- Some computer platforms have emulators that make use of custom compressed formats. We attempt to treat these formats as though they are already ArcBundle files. Notable examples include the GameCube gcm format, the PSP cso format, and the Wii WBFS format.
- Support for common installers is planned. NSIS or MSI will likely be first due to their fairly standardized structure. These should not be considered an alternative to ArcBundles, though, as they only implement a subset of the features available in a proper bundle. As such, when support is added, they will be able to be converted to an arcbundle, but not run directly. This will allow normal arcbundle logic to be added to these programs if necessary.
- Support for AppImages is already bundled into Debian, so no special handling will likely be done here. The format is already near perfect, and we'd just be ruining it if we applied anything else. We plan to support converting between arcbundles and appimages, but as their functionalities differ, not all programs will support this conversion, and it will only apply to Linux applications. We may support an AppImage package manager at some point, but right now we're waiting for someone else to do it. ;)
- Support for other container formats is not planned. ArcBundles are supposed to be a managed installer format, not a container. Managed installers rely on libraries in the host OS, while containers virtualize an OS by overriding some of the host to provide compatible libraries so the program can operate with less regard given to the host OS type or vintage. Attempting to run a container outside of its ecosystem defeats the purpose of a container, as the software would run poorly this way due to library compatibility issues. You could techinically convert a container, but there are no plans to provide a means to do so. Please use a container manager for containers. Rant over.
Mods are stored as regular folders in two locations:
- .arcturus/mods/[bundle name] in the same folder as the archive (this is a hidden folder on Linux, so remember to show hidden folders to see it)
- [HOME]/Sync/Game Saves/mods/[bundle name]
Managed mods are just like regular mods, except each mod's files and folders are stored in an arcmod file, which is a regular arcbundle archive that has a different file extension to prevent programs that search subdirectories from erroneously detecting these files as unique programs themselves. Multiple mod bundles can be stored in the folder, and the locations checked are as follows:
- .arcturus/managedmods/[bundle name] in the same folder as the archive
- [HOME]/Sync/Game Saves/Managed Mods/[bundle name]
Managed mods are unpacked in alphabetical order whenever one changes, and are meant to handle mods where some file collisions may occur. If mod load order matters because of these collisions, you should name these mods so that an alphabetical sort will place them in the correct locations. For example:
This will ensure that files will be overwritten in the proper order. If script-mod-unpack-cleanup-override or script-coremod-unpack-cleanup-override are declared, these will be called after each mod is unpacked to remove common fluff that is unnecessary. This will not automatically handle configuring a program to load these mods in this order, but script-post-mod-install will be called if defined, and script-output-managed-mods will dump the list in the same order they are unpacked. This can be enough for some uses, but script-output-managed-mod-files may be necessary for others. Note that the latter function outputs every file in every mod, so you will likely need to use grep to filter this list to make the mod load config for your program. This function can be called multiple times, and it reads from cached data, so it doesn't need to read every byte of every file every time it is called. This still does not list which mod package each file comes from, as the destination program isn't going to see that data, so it is assumed to not be useful. If that assumption is incorrect, contact us to have additional functions added for your usecase.
To allow the managed mod tools to reuse the pre-packaged mod files that some modders distribute as an archive file, the script-mod-dir command exists. This command will tell the managed mod system to unpack all mod bundles to whatever folder path is specified, relative to the main program folder. "script-mod-dir Mods" will force all mods to unpack to the "Mods" folder inside of the program's main path, and is often enough to allow mods to automatically work when downloaded, renamed to end in .arcmod, and stacked into any of the managed mod folders. Please note that the managed mod tools will attempt to extract any file stored in these folders, so DO NOT put random data files in these folders. You have been warned. Subdirectories of these folders will also be scanned, all in alphanumeric order. You can use folders to sort mods for unpacking priority, but NEVER put anything but mods you want to use in these folders. This does not make subdirectories named after the mod file name, so some games will still need mod package reworking. A future command to correct this is planned.
Packages ending in .arccoremod instead of .arcmod are unpacked to the root game directory instead of the mod folder. These are generally mod loaders, libraries, or game updates. script-coremod-unpack-cleanup-override will get called once for each coremod unpacked, to allow for cleanup of common cruft.
Primer: things to know
The automatic scanners generally do the right thing, and most fixes recommended on WineHQ or other sites aren't needed here. If in doubt, try the program first before attempting to patch it. The following patches are automatically detected in most cases:
- The .NET Framework 2.0 and later
- Some DirectX Library overrides
- Gallium Nine
- MIDI mapper support
- Emulation of the expected version of Windows
- Flash player for swf files
- The Vulkan Runtime
- The Visual Basic runtimes
- Microsoft Foundation Classes
- Games For Windows Live support
- Visual C Runtime 2015+
- XNA Game Libraries
- QuickTime codec dependencies
- Internet Explorer
- RPG Maker redists
- Xaudio2 replacement library FAudio
- cinepak, comctl32, comdlg32, dsdmo, libvorbis
- Windows Media framework 9 and above
Supported for: Windows
Some programs have hard-coded location information in their files or registry entries, and our write-redirection really messes with them. To prevent this issue from occurring, always install the program to C:\prgdir. Before running the program, the wrappers will set this to the correct location of the program, allowing us to still use write redirection through randomly-generated folder names without murdering the program in the process.
Supported for: Windows and Linux
Programs for all computer platforms support a Bash script named arcturus-config in the root of the archive, next to the program itself. For mod or expansion packages scripts stored in a folder named arcturus-config can be used instead, allowing further tweaks to the configuration on a per-mod or per-expansion basis. These scripts are automatically run before the program, and can make folders, apply special patches a program might need that are not typical, or provide additional functionality. This script is generally not required, and as it is simply run as a Bash script, all Bash functionality is available. Some additional commands to control the wrapper are also available for use:
- The command script-change-binary is available to force the wrapper to pick a specific program to run.
- The environment variable scriptWaitFix can be set to a specific file name to force the wrapper to wait for that program to run and then close before assuming everything is done and cleanup can begin. Some programs fork themselves or call another program and terminate, and these programs can get very confused when their program folder disappears because the wrapper thought they were done. This variable should be used to prevent these programs from crashing.
- scriptNoRandomRoot=1 can be set to make a permanent static folder for the write redirection. This method of redirection introduces a bug when the same program is run more than once at the same time, so this should only be used if absolutely necessary. This functionality is used automatically when running a Flash program, as the Adobe Flash player stores saved data relative to the location of the program itself, which really messes up the save files if the program source folder is random.
- script-install-cmds and script-mod-install-cmds are optional bash functions that will be run at the end of installation. Put any code necessary here. If you need to run a windows program for a windows arcbundle, call windows-wine-run with the program to run.
- The command windows-winetricks-install will pass a single package name to Winetricks to be installed without user intervention. This can be used to add a dependency that the automatic scanners miss, or to apply a registry hack that would not be automatically detected. This command will be skipped if already installed, so it will only slow down initial setup, and does not need any additional logic to prevent it from being reinstalled. Please note that this function should only be used as a temporary fix. If using this function fixes your problem, please report the issue to us so we can add a dependency scan to our tools instead. This will repair this issue with ALL affected programs on ALL systems universally, and is obviously the preferred solution. It can also be disabled universally if Wine fixes this issue in the future without requiring recompression of every affected package on every system.
- The variable WINEARCH can be set to win32 or win64. This wine-provided variable sets what type of Windows virtual install is created. It should, where possible, be set to win32 for .NET Framework programs, as the 32-bit .NET Frameworks run better in Wine. The tools automatically choose this setting, and it should not need to be changed. Please contact us with a bug report if this is not the case.
- The variable WINEVER can be set to pick a specific version of Wine. The variable can be set like this: WINEVER=wine-4.0-staging. The version of Wine will automatically be downloaded and installed if needed. Note that using this variable will force the wine version to never update, which could cause the program to break again in the future. If you need this variable, check back periodically as new wine versions are released until it is no longer necessary, then comment it out so you can still refer back to it if it ever becomes necessary again in the future.
- The variable winEngine specifies which Windows emulation engine should be used with this package. Defaults to Wine when no option is recognized. Currently recognized options are wine, mono, or flash.
- The variable xconsole stores the terminal program to use when a Wine program attempts to run in a terminal instead of the usual GUI style. Some Windows terminal programs require this to function correctly - Stardew Valley's mod manager is one example. Usage is xconsole=[terminal program], and is generally xconsole=xterm. This variable should automatically be set, and should usually not need to be changed. If xterm does not function correctly on your system, please contact us so that we can fix the issue.
Windows specific functions:
Supported for: Windows
Windows programs sometimes require registry settings to run correctly. Any necessary registry settings should be exported via regedit to a file named install_data.reg stored in the root of the ArcBundle. This regfile will be automatically imported before the program is run. If you need multiple hives, simply copy the contents of extra regfiles to the first one without the [REGEDIT4] line. This will make a single regfile with all of the necessary changes. For mods or expansions, simply save a reg file in a FOLDER named install_data, and we'll automatically enter each file one at a time before running the main program.
wine_windir - wine_drive_c - wine_program_files - wine_program_files_32
Supported for: Windows
Windows programs also sometimes need to reference files stored in other directories. As these folders are managed by Wine, we need to add some magic to add these files. Simply put them in the correct folder in the main program directory, and Arcturus will take care of copying these files to the correct location. Subdirectories are supported, but care should be taken to avoid adding files provided by Wine or Arcturus to ensure everything works correctly. In other words, if you're not sure if a file belongs here, don't include it.
The folder names and uses are:
- wine_windir - These files are copied directly to the Windows folder
- wine_drive_c - These files are copied to the virtual drive C
- wine_program_files - These files are copied to the "Program Files" folder
- wine_program_files_32 - These files are copied to the 32-bit "Program Files" folder. Usually this is "Program Files" itself, but this differs if a 64-bit wine prefix is used.
Supported for: Windows
Windows programs sometimes need a disc in an optical drive to run. This functionality isn't always well-supported through Wine, so a pair of workarounds exists. This is a folder containing subfolders named drive_d (or drive_e, etc.) that get automatically mounted as virtual optical drives before the program is run. The drive letter is determined by the name of the folder, so drive_d will always be D:, drive_e will always be E:, and so on. Drive letters up to Z can be used, but C and Z are reserved and should never be used for this purpose.
Supported for: Windows
Sometimes the folder approach above just doesn't cut it. For
programs that just don't like the folder method, an ISO image can be
used instead. Put it inside the wine_drives folder as before, then
name it drive_d.iso for D:, drive_e.iso for E:.
Drive letters up to Z can be used, but C and Z are reserved and should
never be used for this purpose.
Yes, we know we said ISO images were limited in scope above, but this format is hard-coded into Wine, so if a program makes use of incompatible data (generally as one of those oh-so-fun copy-protection schemes), the program or its settings may need to be patched to not require a disk at all. As this may be illegal in your country or region (read: if your lawmakers don't understand computers), there may just not be another way. We're looking into integrating some more sophisticated image mounting techniques, but as Linux and Wine generally just fold those down to the data that could be expressed in an ISO anyway, this may not prove to be better.