The Challenge

#

The answer? Really hard. Really, really hard. This post is dedicated to explaining the most important things I’ve learned from this long journey to building my very own package manager, jpkg.

System Design

#

The architecture of jpkg follows a classic client-server model, designed with simplicity and extensibility in mind.

Core Components

#

1. Client Interface

   jpkg install <packagename>
   

2. Package List Management

   A simple way of keeping track of which packages are available in the repository

3. Server-Side Components

   The server handles many aspects of this project, including storing and serving the packages, containing the repository listing, and routing incoming traffic.

Client Functionality

#

The client’s job is simple in theory: Figure out what package the user is talking about and request it from the server. In practice, this requires several additional steps that were surprisingly difficult for me to implement.

Package List Handling

#

The first time the client connects to the server, the client receives a package list.

The packagelist will be saved in ~/.local/share/jpkg/jpkglist

For now, this package list is simply a java array containing Package objects.

However, in the future, I want to use a SQLite database to store these packages. Using a database will relieve tons of issues that a normal array will not. For example, the database will store file paths, checksums, ownership, and installation timestamps which will make uninstalling and checking information on pages sigNIFicantly easier.

After the first request, there are three ways that the client will receive an updated package list.

1. Their package list is too old

If the client’s package list has not been updated in a certain length of time, they will receive a new package list.

2. The package they requested is not on their local copy of the package list

If the client requests a package that is not on their local copy of the package list, it could be on the server’s, so it is requested.

3. The client specifies that they want to update their package list with the cli

   jpkg update
   

Constructing a Request

#

When the client has checked the package list and determined the correct package to request, it sets up a short request to send to the server, containing the requested package and the time since the package list was last updated.

new Request(pack, timeSinceUpdate);

Server Functionality

#

Initially, I thought that the server would be a complete pain to implement. There is so much that it has to do, like getting and caching requests, starting threads to handle multiple requests at the same time, dealing with sending large files over the internet, and figuring out how to store all the packages that it contains.

Oh, you thought I would tell you that it was actually fairly easy? Nope, this time my intuition was correct, and this took a long time. However, I learned a LOT of great skills for the future, so I’m overall glad that I did this project.

Sending the Package

#

I have two ideas for how sending the actual compressed archives of the packages will work. My first idea is that I can just use DataInput and Output Streams. However, if this proves to be infeasible, I think that the http protocol will be perfect for sending these large files, just like cURL does.

After the package is sent, the it should be verified with a checksum to make sure it hasn’t been tampered with in transit.

Installing the Package

#

Once the package is received, a shell script that is shipped with the package is run to compile and install any code into the right places on the user’s system.

THIS IS TERRIBLE. This is a ridiculous way to do this, and I need to make a package manifest as soon as possible, most likely in Yaml, but again, there is a large amount of learning that I will have to do.

I might also dabble in sandboxed installation, sending over HTTPS, and adding package signatures.

Code

#

Core Classes

#

jpkg’s core is based heavily around the Package and Request classes. These take on a lot of the organizational burden, so it’s worth mentioning them.

The Package Class

#

The Package class represents a single installable unit in the system. Each package needs to track its basic metadata and dependencies:

class Package {
    private String name;
    private String version;
    private String description;
    private Package[] dependencies;
    private int id;

    public Package(String name, String version, String description, Package[] dependencies, int id) {
        this.name = name;
        this.version = version;
        this.description = description;
        this.dependencies = dependencies;
        this.id = id;
    }
    
    // lame getters and setters go here
}

Each package contains:

A unique name and version

A short description of its functionality

An array of Package objects it depends on

A numeric ID for quick lookups (This will be replaced by a hashtable solution in a later release)

The Request Class

#

When a client needs to fetch a package, it constructs a Request object:

class Request {
    private Package pack;
    private int timeSinceLastUpdate;

    Request(Package pack, int timeSinceLastUpdate) {
        this.pack = pack;
        this.timeSinceLastUpdate = timeSinceLastUpdate;
    }
    
    // guess what i omitted here
}

The Request class is intentionally simple - it just needs to track what package is being requested and when the client last updated their package list. This helps the server decide whether to send an updated package list along with the requested package.

The ListManager Class

#

When the client needs to work with the package list, like updating it, checking against it, and more, a ListManager will be created to

The ClientManager Class

#

Functionality

#

Alright, now that we’ve set up some classes we’ll need, let’s get something working.