25

I'm creating a data storage device that takes a certain number of pictures of the night sky over a couple of hours, and the pictures will be downloaded right after they are all taken. The memory card must be able to store all of the pictures at once.

The JPEGs that will be taken are 640x480 pixels, and it is essential that there is enough room on the memory card for all 100 of them. So what is the largest size of a 640x480 JPEG?

I have taken some test pictures to figure this out:

#1#2#3

  • The file size of the "stackoverflow" image is 73,774 bytes.
  • The file size of the white image is only 36,607 bytes.
  • But the file size for the checkered photo clocks in at 149,339 bytes.

I am assuming that the file size increases with complexity.

How can I build enough room on the memory card to fit 100 640x480 JPEGS, without knowing how complicated and what size they will be? I don't want to waste extra space as I may be making many of these capture devices.

Journeyman Geek
  • 127,463
  • 52
  • 260
  • 430
Blue Ice
  • 547
  • 1
  • 7
  • 18
  • it depends on the image producer. 100-quality JPEG can easily blow up in size. What is your camera and camera settings? – John Dvorak Aug 25 '13 at 04:22
  • The test camera is a Canon Powershot A1100 IS. For more info, you can check the metadata, because I'm not sure what you are asking for. – Blue Ice Aug 25 '13 at 04:25
  • JPEG file size depends on the **Q** (quality) setting. At a Q of 100, the file size for a 640x480 image would be close to the raw file files of 921600 bytes. – sawdust Aug 25 '13 at 04:26
  • "exif not found", possibly due to the upload process – John Dvorak Aug 25 '13 at 04:34
  • Oh. What do you think I should do, then? – Blue Ice Aug 25 '13 at 04:36
  • 1
    HAve you taken any sample pictures of the night sky at dif. Q settings? As a test? – Carl B Aug 25 '13 at 04:59
  • 2
    What is this _for_? Are you sure jpeg is the right choice of format. – Jack Aidley Aug 25 '13 at 07:55
  • 3
    The add some background to Jack Aidleys comment: JPEG compression changes the picture. It makes assumptions and discards information in order to get a smaller file size. (Unless set to 100% quality, in which case you might as well use an uncompressed format. Often a digital camera has a tiff or even a *raw* setting for this. If you want to keep all small dots such as stars then use one of these). This will also make the size of an image completely predictable. – Hennes Aug 25 '13 at 09:57
  • Is that CRT monitor I see on images? Also, width*height*4 + header is rough estimation for image size. – Luke Aug 25 '13 at 14:37
  • It's an LCD :) I don't have CRTs anymore... – Blue Ice Aug 25 '13 at 20:00
  • If the images are photos with lots of gradients, jpg is fine. If they are limited color artwork like the first and last images, jpg is a terrible format. It will introduce artifacts. If what you're capturing is the night sky and what you're interested in are bright stars against a dark sky, jpg isn't a great choice. If you're stuck with that, select the lowest compression setting available and use a larger storage device. – fixer1234 Mar 24 '17 at 22:00

2 Answers2

35

Just to check, let me test ForeverWintr's analysis experimentally.

The worst kind of input image for JPEG compression (or any compression, really) is uniformly random RGB noise, which is theoretically incompressible. So let me generate some using the netpbm tools:

$ rawtoppm < /dev/urandom 640 480 > rnd.ppm
$ pnmtopng < rnd.ppm > rnd.png
$ du -b rnd.*
923772  rnd.png
921615  rnd.ppm

Uniformly random RGB noise, lossless PNG format
(Uniformly random RGB noise, lossless PNG format, 903 kb)

Note (March 2017): I'm fairly sure the image above was in PNG format when I first wrote this answer and uploaded it back in 2013. (There's even a comment about color management below that strongly implies this.) Unfortunately, it would seem that it has been silently converted into JPEG at some point, making the visual comparison here useless.

I've tried to reupload a new PNG test image, but apparently it hits some kind of an arbitrary PNG file size limit at imgur and gets auto-converted to JPEG. I'm not sure if there's any way around this issue, but at least if you have access to a Linux box, you can always re-run the given commands to generate your own test images. In any case, other than preventing direct visual comparison of the compression quality, this does not invalidate the analysis below in any way.

OK, so the uncompressed PPM file is 640 × 480 × 3 = 921,600 bytes long, plus 15 bytes for the minimal PPM header, just as expected. Trying to losslessly compress it using the PNG format just ends up increasing the size by 2157 bytes, presumably taken up by PNG headers and metadata and possibly some slight inefficiency in the compression algorithm trying to compress incompressible data.

(Yes, that's 3 bytes per pixel, not 4; even the PPM format, which is about as simple as a graphics file format can get, isn't dumb enough to store a useless fourth byte per pixel on disk. There may be some advantage to doing so in memory for alignment reasons, especially if you also need to store an alpha channel, but those reasons don't apply when writing the image to a file.)

OK, so what about JPEG? Let's try to minimize the compression losses first (quality = 100, no chroma subsampling, floating-point DCT). Unfortunately, the pnmtojpeg manual doesn't clearly explain how to set all the relevant options (specifically, the -sample option is listed in the "Options for wizards" section, which just references a file in the libjpeg documentation), so I'll convert it in the GIMP instead. The resulting file looks like this:

897249  rnd.jpg

JPEG compressed RGB noise, quality = 100, no chroma subsampling
(JPEG compressed RGB noise, quality = 100, no chroma subsampling, 876 kb)

What, how can it be smaller? Didn't I just say pure noise was incompressible? Well, the thing is, even at maximum quality, normal JPEG compression isn't quite lossless. Reopening the image in GIMP and comparing it with the original, one can see that some pixels have had their color values shifted by one or two steps (out of 256). Those are the pixels where the JPEG compression algorithm "cheated" and threw away a bit here, another there, where it estimated that the change wouldn't be noticeable. Indeed, to the unaided human eye the result is quite indistinguishable from the original, but those discarded bits do add up to a measurable decrease in file size, even after accounting for header and encoding overhead.

So that was maximum quality; what about more typical settings, like the pnmtojpeg defaults (quality = 75, subsampling enabled)? Let's try it:

$ pnmtojpeg < rnd.ppm > rnd2.jpg
$ du -b rnd2.jpg
185128  rnd2.jpg

JPEG compressed RGB noise, quality = 75, chroma subsampling
(JPEG compressed RGB noise, quality = 75, chroma subsampling, 184 kb)

Wow, from 901 down to 184 kb! That's pretty aggressive compression, though, and you can definitely tell the difference when comparing the images closely. Most of it's because of the chroma subsampling, which basically just throws away 75% of the color (hue / saturation) data. Trying it in the GIMP with subsampling disabled gives a 350,618 byte file that still looks (to the human eye, at least) pretty close to the original even when magnified.

Anyway, the point of all this is to demonstrate that, no matter how noisy your night sky photos might be, and no matter how high a quality you might select, there's just no way a 640 × 480 JPEG file can get significantly larger than 900 kb. (Well, unless your camera attached a multi-megabyte Exif color profile to it or something equally stupid, that is.) And if you're using more typical JPEG compression settings, the maximum plausible file size goes down to about 200 kb or so.

Ilmari Karonen
  • 1,648
  • 14
  • 21
  • 4
    *theoretically incompressible* only for lossless compression though, right? – Daniel Beck Aug 25 '13 at 12:56
  • 1
    @DanielBeck: Right. Obviously, you can compress any data as much as you want if you're willing to just throw away parts of it. (That's basically what JPEG compression does, it just tries to do it in such a way that the lost parts are not noticeable to the human eye, and that what remains can be encoded compactly. Noise is still a tough case for it, since the _only_ thing even a lossy compression algorithm can do with noise is throw away parts of it.) – Ilmari Karonen Aug 25 '13 at 14:13
  • Maybe that's just me, but the second image looks brighter than the first one. – Bogdacutu Aug 25 '13 at 16:12
  • @Bogdacutu: It shouldn't, although it's always possible that your browser may be doing something weird with color management etc. Try loading them both in a graphics editor and comparing the color values. – Ilmari Karonen Aug 25 '13 at 16:17
  • Nice writeup @Ilmari. – ForeverWintr Aug 25 '13 at 16:41
22

Here I suggest an upper bound for JPEG file sizes. See Ilmari Karonen's answer for a discussion of more typical jpeg sizes.

The pixel storage space for a 640X480 32-bit bitmap image can be calculated like so (based on this answer, but corrected based on Ignacio Vazquez-Abrams' comment and this answer):

Assuming that no compression has been applied to the file, there are 307,200 pixels, which is a 0.3MP. Handy look up table

If each pixel contains 32 bits of information, then

  1. 307,200 * 32 = 9,830,400 bits of information
  2. Divide by the 8 bits to become a byte value
  3. 9,830,400 / 8 = 1228800 bytes (Or 1.17 Mb)

This is the size of an uncompressed bitmap, and as such should be an upper bound for jpeg file size (In reality, because the JPEG format uses compression, your images should be much smaller, especially given that you're taking pictures of the night sky, which I imagine contains a lot of black. Note that the largest example image in your question is only 0.14 MB).

With regard to your specific issue however, even using that upper bound, 100 images is only 117 MB, and it's been a long time since I've seen a memory card as small as 128 MB. I suspect any currently available memory card will have enough capacity to meet your needs.

Apparently the issue of maximum jpeg file size is subject to some debate. This Stack Overflow answer suggests a theoretical maximum size of 20.25 bytes per pixel, or 5.9 MB in your case, but producing an image of that size requires deliberate misuse of the jpeg format's compression scheme, so it's extremely unlikely that you'd ever see such a thing produced by a camera.

ForeverWintr
  • 879
  • 5
  • 14
  • 1
    Unfortunately even the uncompressed bitmap value is just a bit low, since it assumes packing. An unpacked bitmap uses 32 bits per pixel (for [alignment](http://en.wikipedia.org/wiki/Data_structure_alignment) reasons), increasing file size by 33%. – Ignacio Vazquez-Abrams Aug 25 '13 at 05:40
  • Thanks @IgnacioVazquez-Abrams. That's what I get for assuming an accepted answer is correct. – ForeverWintr Aug 25 '13 at 06:11
  • 1
    @IgnacioVazquez-Abrams - "Alignment" is an attribute dictated by the processor (not the storage medium), and is convenient when the data is in RAM for processing. For storage purposes, a 32-bit word alignment is not a necessity and certainly an extravagance. Packing data is a common operation prior to writing to storage, especially for a certain 25% savings. I've seen digital cameras that store each pixel in exactly 3 bytes for the raw format. – sawdust Aug 25 '13 at 06:15
  • 1
    "... certainly an extravagance." Nowadays. Many moons ago it used to be considered a cycle-saving feature (no need to align on load, given how long it would take). – Ignacio Vazquez-Abrams Aug 25 '13 at 07:59
  • Also if you deal with specialized systems the packing could be done on-chip without any cycle loss (drop every 4th byte is trivial to implement in specialized unit and might be worth the effort if the operation is common). – Maciej Piechotka Aug 25 '13 at 10:00
  • 3
    In practice, while some systems might store RGB image data as 4 bytes per pixel _in memory_, pretty much all image _file_ formats use 3 bytes per pixel at most (unless there's an actual alpha channel stored in the fourth byte). See my answer below. – Ilmari Karonen Aug 25 '13 at 12:56
  • -1 for editing your answer to accommodate an irrelevant comment. Data storage does not use or require 32-bit alignment. And *"many moons ago"* storage capacity was a premium and transfer speeds were slow, so data packing was common practice. Wasting 25% of storage space and increasing transfer times 33% is poor storage management. – sawdust Aug 25 '13 at 23:59
  • @sawdust, the question was about the **largest size** a 640X480 jpeg image could conceivably be. Based on both Ignacio's comment and the answer I quoted, I decided to go with the more conservative (larger) estimate of bmp storage. If you disagree that a 640X480 image can be this size, provide a reference. – ForeverWintr Aug 26 '13 at 02:21
  • *"If you disagree that a 640X480 image can be this size..."* - How did the question change from a JPEG image to *"an image"*? @IlmariKaronen has offered empirical results that 3 bytes per pixel is an upper bound for a JPEG filesize. So you rather choose an overestimation of 33% than a reasonable size? Even at the highest JPEG quality setting, the camera will probably use a Q of only 70 to 80. So your "answer" actually fails the OP's requirement of *"I don't want to waste extra space"*. If I had a solid reference then I would have written an answer instead of the 3rd comment to the question. – sawdust Aug 26 '13 at 23:33
  • @sawdust. My reasoning: 1) uncompressed bmp is a reasonable upper bound for the size of a jpeg of the same resolution, as it's an image stored simply as pixel values. Note that upper bound says nothing about *average* or *typical* values. 2) I was corrected about the maximum size of a bmp image. My own reading corroborated this correction. 32bpp bmps exist, and a brief image search confirms that 900kb is not an upper bound for bmp size. 3) Arguments about wasted space are of dubious value, because the smallest amount of memory you can buy for a modern camera is larger than my high estimate. – ForeverWintr Aug 27 '13 at 05:44
  • My responses: #1 & #2: bmp filesizes are not relevant. You have not established a correlation between JPEG and bmp. #3 means that your answer is not scalable. If the image sensor is 8 megapixels instead of 300K, then a 33% overestimation is disastrous. Actually you're entitled to err. I'm more surprised by the OP accepting this answer/guess rather than Ilmari's reasoning. – sawdust Aug 27 '13 at 06:05
  • @sawdust. I have established a connection between jpeg and bmp (not a correlation, nor have I attempted to). See my explanation in my answer, the linked answer, and my explanation to you an hour ago. You can dispute my reasoning in doing so, but simply stating "bmp filesizes are not relevant" is not a reasonable argument. I agree that Ilmari's answer is excellent, but I don't find it antithetical to mine. He demonstrates typical large jpeg sizes, I estimate the largest possible (and note that Ilmari uses PPM, assumably for the same reason I used bmp--it's a very basic image storage format). – ForeverWintr Aug 27 '13 at 06:53