There’s a class of packaging bug that doesn’t show up in your test suite, doesn’t appear in CI, and doesn’t exist on Ubuntu. It manifests as a single systemd exit code — 203/EXEC — and it means your binary never started. Not “started and crashed.” Never started. The kernel refused to execute it.

The culprit is an SELinux file context label, and the fix requires understanding where those labels come from.

The Symptoms

Two installations of the same software on the same Fedora host. One installs to /opt/myapp/bin/myapp. The other installs to /srv/myapp/releases/20260401/bin/myapp. Same binary. Same systemd unit file. One runs. One doesn’t.

The working install:

Process: 243952 ExecStart=/opt/myapp/bin/myapp
         (code=exited, status=1/FAILURE)
Mem peak: 93.7M
CPU: 3.927s

Exit code 1. The process ran for four seconds, consumed 94MB of memory, did real work, and failed for an application-level reason (in this case, a network connection refused). That’s a normal failure.

The broken install:

Process: 243338 ExecStart=/srv/myapp/releases/20260401/bin/myapp
         (code=exited, status=203/EXEC)
Mem peak: 1M
CPU: 5ms

Exit code 203. One megabyte of memory. Five milliseconds of CPU. The process never came into existence. Systemd attempted to exec the binary and the kernel said no.

“But I Can Run It From My Shell”

Here’s where the debugging goes sideways. You SSH into the box, run the binary directly, and it works fine:

bash

$ /srv/myapp/releases/20260401/bin/myapp --version
myapp 2.4.1

No error. No denial. So you conclude the binary is fine and start investigating systemd unit file syntax, PATH issues, missing libraries — everything except the actual problem.

The reason it works from your shell is that you’re running as an unconfined user. When you log in interactively on a RHEL/Fedora system with targeted policy, your shell runs in the unconfined_t domain. Unconfined users have execute permission on essentially all file types, including default_t. The entire point of the “unconfined” domain is that SELinux doesn’t restrict it — it’s the escape hatch for interactive administration.

But systemd services don’t run in unconfined_t. A service without a dedicated SELinux policy module runs in unconfined_service_t (or init_t, depending on the configuration). These domains are more restrictive than a login shell. They can execute bin_t, shell_exec_t, and other standard executable types. They cannot execute default_t.

This is the matrix that matters:

So the binary “works” because you’re testing it from the one domain that doesn’t care about the label. The moment execution shifts to a service context, the label matters. Other scenarios where default_t will block you include cron jobs (running in crond_t or system_cronjob_t), scripts called from confined daemons like httpd or sshd, and anything launched via at, batch, or systemd-run.

The diagnostic shortcut: if it runs interactively but fails as a service with 203/EXEC, check ls -Z on the binary. If you see default_t, you’ve found your problem.

What Happened

Check the SELinux file context on each binary:

bash

$ ls -Z /opt/myapp/bin/myapp
system_u:object_r:bin_t:s0  /opt/myapp/bin/myapp

$ ls -Z /srv/myapp/releases/20260401/bin/myapp
unconfined_u:object_r:default_t:s0  /srv/myapp/releases/20260401/bin/myapp

The difference is the type field — the third component of the SELinux context. One is bin_t. The other is default_t.

bin_t is the SELinux type for system executables. Most domains — including the unconfined service domain that systemd uses for services without a dedicated policy — have execute permission on bin_t. Your binary runs.

default_t is SELinux’s way of saying “I have no idea what this file is.” Red Hat’s documentation on the file_t and default_t types is explicit: default_t is assigned to files in directories that have no matching file context rule. It is not a permissive fallback. Most domains do not have execute permission on it.

When systemd tries to exec a default_t binary, SELinux denies the operation at the kernel level. The process never loads. You get 203/EXEC.

Where bin_t Comes From

The natural question is: why does the /opt binary get bin_t automatically? Nobody ran semanage fcontext for that path.

The answer is in the SELinux Reference Policy — the upstream policy source that RHEL and Fedora both derive from. Specifically, in corecommands.fc, the file context definition file for core system commands. Under the /opt section:

/opt/(.*/)? bin(/.*)? gen_context(system_u:object_r:bin_t,s0)
/opt/(.*/)? libexec(/.*)? gen_context(system_u:object_r:bin_t,s0)
/opt/(.*/)? sbin(/.*)? gen_context(system_u:object_r:bin_t,s0)

These are regex rules. The first one says: any path under /opt, at any nesting depth, that passes through a directory named bin, should be labeled bin_t. So /opt/myapp/bin/myapp matches. So does /opt/whatever/some/deep/path/bin/thing. The pattern assumes that if you’re putting something in a bin directory under /opt, it’s an executable, and it should be treated as one.

You can verify this on a running system:

bash

$ matchpathcon /opt/myapp/bin/myapp
/opt/myapp/bin/myapp    system_u:object_r:bin_t:s0

$ semanage fcontext -l | grep '/opt.*bin'
/opt/(.*/)?sbin(/.*)?    all files    system_u:object_r:bin_t:s0
/opt/(.*/)?bin(/.*)?     all files    system_u:object_r:bin_t:s0

The corecommands module also defines rules for /usr/bin, /usr/sbin, /usr/libexec, and various paths under /etc that contain scripts (cron directories, network-scripts, profile.d, and so on). It’s a large file — 471 lines — and it constitutes the kernel of SELinux’s understanding of “what is an executable on this system.”

RPM-based package managers participate in this system automatically. When RPM installs files, it runs restorecon as part of the transaction, which reads these rules and applies the matching labels. If your RPM installs a binary to /opt/myapp/bin/, it gets bin_t for free. No scriptlet needed.

Where default_t Comes From

default_t comes from the absence of a matching rule.

The /srv directory, the /hab directory, a custom /deploy directory — if the SELinux policy has no file context regex that matches the path, the file gets default_t. As the DBI Services blog on SELinux setup and configuration notes, this is a common stumbling point for software installed outside standard paths. The binary is perfectly valid, the filesystem permissions are correct, and SELinux still blocks execution — not because of a deny rule you wrote, but because of the absence of a permit rule nobody thought to add.

This is by design. SELinux’s default posture is denial. default_t is not a bug; it’s the policy working correctly. The bug is in the packaging.

The Fix

The immediate fix is straightforward:

bash

semanage fcontext -a -t bin_t "/srv/myapp/.*/bin(/.*)?"
restorecon -Rv /srv/myapp

semanage fcontext -a adds a new file context rule to the local policy store (stored in file_contexts.local, which takes priority over the base policy rules). restorecon then walks the filesystem and applies the correct labels based on all active rules.

This survives reboots and filesystem relabels. It will not be overwritten by policy package updates because local modifications have higher priority.

For packaged software, the right place to do this is in the package’s post-install script — an RPM %post scriptlet, a DEB postinst, or whatever hook your packaging system provides. The pattern is:

bash

if command -v getenforce >/dev/null 2>&1 && [ "$(getenforce)" != "Disabled" ]; then
  semanage fcontext -a -t bin_t "/srv/myapp/.*/bin(/.*)?" 2>/dev/null || true
  restorecon -Rv /srv/myapp/bin 2>/dev/null || true
fi

Guard on getenforce so you don’t fail on systems without SELinux. Swallow errors so you don’t block installation on systems where policycoreutils isn’t installed.

The Deeper Lesson

There are two distinct SELinux policy mechanisms that software maintainers confuse:

Type Enforcement (TE) policy — .te files compiled into .pp modules — defines what a running process is allowed to do. It creates domains, grants permissions, defines transitions. If your application needs to read files it normally couldn’t, or listen on a non-standard port, you write TE policy.

File Context (FC) policy — .fc files or semanage fcontext rules — defines what label a file on disk should have. It maps path regexes to SELinux contexts. If your binary can’t even be executed, the problem is almost certainly here.

These are different layers. A TE module that grants your application runtime permissions does nothing if the binary is labeled default_t and can’t be exec’d in the first place. You need the FC rule before the TE rule matters.

I’ve seen packaging pipelines that ship a carefully crafted .te module granting read access to various file types, while completely omitting the .fc rules that would let the binary execute. The TE module is solving a problem that can’t even be reached yet. It’s like writing detailed error handling for a function that never gets called.

Checking Your Work

A few commands that are worth memorizing:

bash

# What label SHOULD this path have, according to policy?
matchpathcon /opt/myapp/bin/myapp

# What label DOES this path have right now?
ls -Z /opt/myapp/bin/myapp

# What rules exist that might match this path?
semanage fcontext -l | grep myapp

# What local customizations have been made?
semanage fcontext -l -C

# Fix labels on a path to match policy
restorecon -Rv /opt/myapp

If matchpathcon returns default_t for your binary’s path, you need to add a file context rule. If matchpathcon returns bin_t but ls -Z shows default_t, you need to run restorecon. If both agree on bin_t and your binary still won’t exec, you have a different problem (check ausearch -m AVC for the actual denial).

Summary

The SELinux reference policy ships with file context rules that automatically label binaries under /opt/.*/bin/ as bin_t. This is defined in corecommands.fc in the upstream reference policy and inherited by every RHEL and Fedora system. Software installed to paths covered by these rules — including anything under /opt with a bin directory — gets correct labels automatically.

Software installed to paths not covered by these rules gets default_t, which is not executable. The fix is a file context rule added at package install time. The diagnostic is ls -Z and matchpathcon. The exit code to watch for is 203/EXEC.

If your packaging system doesn’t account for SELinux file contexts, your software works on Ubuntu and breaks on Fedora. That’s not a Fedora bug. It’s a packaging gap.