Hardware ramblings from a software guy

My middle daughter asked me a couple of times how a photo is produced in the camera. I didn’t know what to answer, there is too much physics I don’t understand.

But the rough picture is relatively simple. Light hits an image sensor, a silicon chip covered in millions of tiny photodiodes. Each one turns photons into an electrical charge and those charges become digital pixel values. From there the data leaves the sensor and travels through the rest of the camera:

light ──► silicon sensor ──► ISP ──► YUV frames ──► encoder ──► MP4

• The ISP (image signal processor) turns the sensor’s raw readings into a viewable image: demosaicing, noise reduction, tone mapping.
• The result is YUV frames: the working pixel format on the bus before compression.
• The encoder compresses those frames into H.264/HEVC and packs them into an MP4. That step is where most of the size disappears: the same footage that is hundreds of megabytes per minute as raw pixels typically becomes a few to tens of megabytes per minute as a normal video file (often roughly tens to hundreds of times smaller).

Digital signatures

If we want to prove later that a photo or video came from this camera, and not from a deepfake generator, we digitally sign it. But we don’t sign the megabytes of pixels directly: we first hash them into a short fixed-size fingerprint, then sign that fingerprint with a private key that never leaves the device.

But where in the pipeline should we hash?

light ──► silicon sensor ──► ISP ──► YUV frames ──► encoder ──► MP4
                                      ▲
                               where should we hash?

In the last few days I have been doing the homework of a software guy who suddenly needs to care about camera silicon. The question is where the hash should be computed. Before ISP? Before YUV frames? Before or after encoder? Also, what should be hashed: Bayer data, YUV frames, or the finished MP4? Before the ISP the sensor typically outputs raw Bayer data (a mosaic of single-color samples, one per light-sensitive cell on the sensor). The ISP turns that into viewable frames, usually YUV. The encoder then turns those frames into a compressed bitstream inside an MP4.

The ISP and the hardware encoder are usually fixed-function blocks on the same SoC (system-on-chip: the main chip that integrates CPU, ISP, encoder, and often security hardware), so they are parts of the silicon. The path between them is typically an on-chip bus. That means wires inside that chip that carry data from one block to another, not an external connector you can attach to. That does not mean we can simply trust them. An attacker who cannot tap that bus directly might still try to change what the ISP outputs by compromising firmware or configuration, if the device allows that. Whether that is realistic depends on secure boot and how locked-down the ISP/encoder firmware is.

But note that despite not being perfectly secure, hashing on the chip is still a much better option than hashing in ordinary software. An attacker who controls an application can hand it fabricated frames, and those frames get hashed and signed as if they came from the camera.

A small dedicated hash circuit on the SoC is safer because it hashes what actually flows between the camera blocks, not frames a potentially compromised application prepared.

So ideally, we would need a small dedicated circuit on the SoC to compute the hash.

I was wondering whether manufacturers already do this. Some cameras already hash and sign at capture (Leica, Pixel). That is not the same as a dedicated hash engine on the SoC that fingerprints the pixel stream before software can touch it.

Today’s common pattern is to compute the content hash later, in an application or in a TEE fed by the SDK. Public details are frustratingly scarce. What I could find is a Sony patent application that describes signing image data (or a processed version of it) with a key tied to the image sensor. It does not describe a full public product pipeline (exactly which stage is hashed, in which cameras, with which hash), so it is hard to tell from the patent alone what ships today.

Another approach is the MIPI Camera Security Framework (aimed mainly at automotive CSI-2). It focuses on authenticating a secure sensor and protecting the sensor-to-SoC link, rather than on a hash block later in the pipeline for edit proofs.

The cost: what do you have to store?

The question is not only where to hash, but also what to hash: Bayer data, the YUV frames on the bus, or the finished MP4? Bayer is the earliest option, but edit proofs need the viewable frames, so the practical choice is YUV versus MP4.

But what is YUV actually? We are more used to thinking in RGB: each pixel is a mix of red, green, and blue. Cameras and codecs usually use YUV instead. Y is brightness; U and V carry color. That arrangement exists because human vision cares more about brightness than about fine color detail, so U and V can be stored at lower resolution than Y without looking much worse.

The encoder’s job is to shrink YUV frames into the small file you normally keep and play. Ordinary H.264 reduces the size a lot and loses information in the process: decoding the MP4 does not recreate the exact bytes that were on the bus.

The thing is that proving the edits is significantly easier when we hash the YUV frames instead of a smaller MP4. Eva-style edit proofs (see Eva: Efficient Privacy-Preserving Proof of Authenticity for Lossily Encoded Videos) operate on macroblock YUV pixel witnesses, not on an MP4 bitstream: the circuit checks that an edit gadget was applied to those pixels and that the hashes match.

However, uncompressed planar YUV takes ~700 MB/min at 1080p (Full HD, 1920×1080), while a normal H.264 file is often tens to hundreds of times smaller. So if we want faster proofs that bind to pre-encode frames, we will need considerably more storage than for a normal MP4.

Summing up

So, what do we need to enable edit proofs?

Ideally, a hash circuit on the silicon. And what is to be hashed is not the MP4, but YUV frames.

The first is hard to expect from manufacturers soon, because it requires chip modifications. The second can be done without that: an SDK callback (or post-capture tooling) can already expose or retain pre-encode YUV, at the cost of storage and a weaker trust story until the hash itself also moves into silicon.

EDIT (2026-08-07): hashing on the chip is not enough by itself

A friend read the earlier claim that hashing on the chip is safer than hashing in ordinary software, and commented that even if you hash on the chip, the signature still happens in a TEE-like environment. If an attacker breaks into that signing processor and gets the private key, they can sign whatever they want, regardless of where the hash was computed.

My friend is right. We also need to enforce that the signer uses that hash.

The device is divided into two parts:

• Normal world: the main OS, the camera app, and most of the camera SDK.
• Secure world (the TEE, trusted execution environment): a hardware-isolated corner of the same chip where small Trusted Applications (TAs) run. The private key used to sign at capture should live here (or in a secure element) and never be readable by the normal world.

Hashing on silicon only helps against app-level forgery if the signer is bound to the digest produced by the on-chip hash circuit. In practice that means roughly:

1. Digest in memory the normal world cannot forge. The hash engine’s output should land where only the TEE can read it (for example a register or secure on-chip RAM).
2. The signing API is narrow. The normal world should call something like “attest / finalize this capture,” not “here is a digest, please sign it with the device key.” A dedicated TA reads the hardware digest and signs; that key should be usable only by that TA, not as a generic “sign any blob” service for apps.

If the second point is missing (if software can still submit an arbitrary digest to be signed with the device key), then on-chip hashing does not help much.

But with that binding in place, a compromised camera app should no longer be able to invent a digest and get a valid capture signature. Breaking into the TEE or secure element and extracting or misusing the device key is a different attack, and that one still works.

Where does this leave us?

Putting the digest itself beyond the normal world’s reach (hash engine output only the TEE can read) needs hardware support. If a hardware partner asked what to implement, the answer would be roughly three things most chips do not offer as an open camera-attest path today: a hash engine fed from the camera pixel path, a place for that digest that only the TEE can read, and wiring so the device key can sign that digest and nothing else. The first two are silicon or SoC-integration work. The third is mostly secure firmware: the small trusted programs in the TEE (the TAs) that are allowed to read that digest and call the device key, and that refuse to sign anything else.

A concrete example of hash digests sitting in protected state is a TPM’s Platform Configuration Registers (PCRs). Those registers hold measurement hashes (of boot software and configuration, not of camera frames) that ordinary software generally cannot overwrite with an arbitrary value; software can only extend them (fold in a new hash). Later the TPM can quote those PCR values: produce a signature over them with an attestation key, so an outsider can check that those digests really came from that TPM. The content differs from capture hashing, but the pattern is close: digests held where normal software cannot forge them, then a privileged attest/sign step.

I find it frustrating how little of this is spelled out in public. Vendors say things like “secure ISP pipeline,” “sealed in the TEE,” or “sign before OS.” That is consistent with a digest only the TEE can read, but also consistent with other designs, for example the TEE hashing buffers it received, hardware copying pixels into memory along a path the normal world cannot tamper with, or a vendor black box. From outside, we do not know which mechanism they use. Some products’ marketing is compatible with the harder hardware piece above, but public detail is not enough to confirm it.

EDIT (2026-08-11): a design sketch

The rambling above asked what a hardware partner would need to build. I have since sketched a design: a locked pixel tap into a binding engine, forge-proof registers for the binding value, and a secure element that signs only that value. The same shape can cover Eva-style hash binding and zk-Cinema-style commitment binding, so a new proof system should preferably need new configuration, not a new chip.

The short version is on the Hardware page; the longer write-up is the hardware requirements for capture binding.