CHDK and very long exposure

Is it possible to use the CHDK to make a Canon Powershot S3 IS take a
picture with an exposure time of half an hour? Having to write a
custom script would not be a problem.

On Tue, 02 Oct 2007 14:54:47 -0700, "robotiser@googlemail.com"
<robotiser@googlemail.com> wrote:

>Is it possible to use the CHDK to make a Canon Powershot S3 IS take a
>picture with an exposure time of half an hour? Having to write a
>custom script would not be a problem.

No, that would be a firmware modification. CHDK does not touch the camera's
firmware and is not firmware (so it also voids no warranties). Even if it were
possible then you would be faced with excess noise at such long shutter speeds
from one exposure. However this doesn't mean that such a thing is impossible
with your camera, it'll just take a little extra work on your part. The results
beating any other camera's on earth, making it worth the extra effort.

What you do is use any of the CHDK intervalometer scripts with some
pre-planning.

Go into CHDK's RAW menu options and turn Noise Reduction to OFF. This disables
the dark-frame noise-reduction routine that happens on all shutter speeds longer
than 1 second. Instead of having an 18 second gap between exposures (the time
needed by the camera to do a dark-frame subtraction on a 15-second exposure),
you'll now only have a little over a 1 second delay between shots. Keep in mind
too that this affects JPG files as well as RAW, whether you are saving RAW files
or not.

Set your intervalometer script to shoot in continuous burst mode for as long as
you need for your whole exposure. If using an S-series camera then I recommend
the OMNI-Intervalometer script as that has the option to use the camera's own
burst mode. For others who are not using that script with an S-series camera
then you'll have to use one of the other intervalometer scripts in non-stop mode
using a very short delay between shots, or write your own script with the
appropriate commands to use its burst mode.

During the shooting session, every few minutes or so, you'll have to record 1
dark-frame by capping or covering the lens so you can later do dark-frame
subtraction manually using something like this freeware "BlackFrame NR" utility
from http://www.mediachance.com/digicam/blackframe.htm Covering up the lens
every now and then is known in astro-photography circles as the "hat-trick
technique" because a hat really was used to cover up the larger size of the
telescope's optics. Being able to hold the hat apart from touching and
disturbing the telescope and camera while still blocking out all light for a
dark-frame exposure.

The number and interval of dark-frames that you'll have to record will depend on
the length of your photo session, the ambient air-temperature (warmer weather
means a warmer sensor means more thermal-noise in sensor; longer sessions means
warmer sensor too), and the quality of the sensor that was used to build your
camera. Sensor quality varies from camera to camera, even the same models built
on the same day on the same production line. You might have lucked-out and
obtained a very quiet one. You will probably be able to get by in most
situations by recording 1 dark-frame every 5 to 15 minutes worth of 15-second
exposures. Recently I did just fine with only 1 dark-frame taken at the end of
20 minutes worth of 15-second exposures. I compared the use of earlier
dark-frames with using just the last one and there was no difference.

Later you can combine all these 15-second exposure images into one long exposure
image using a freeware utility called Registax from
http://www.astronomie.be/registax/ There are others but this is one of the
better ones.

Using this method you are able to get noise-free images at any ISO for any
exposure length that you desire. Yes, I typed that correctly, at any ISO. Noise
averages out and disappears when using frame-stacking methods with enough
photos. That's also why this technique is so popular in astronomers' circles for
decades. Getting exposures of this length is impossible to do with any other
high-resolution digital cameras using just its own capabilities, unless tethered
to a bulky and battery-time-limited laptop to control it. Today this method is
still being done with web-cams and video cams but at greatly reduced
resolutions. They're the only other devices that had the capabilities needed,
that which CHDK now adds to a high-resolution still-frame camera.

Pretty cool, eh?


(Side info: the reason the "Noise Reduction" option ended up in the RAW menu is
that originally there was a problem with the camera's dark-frame routine in the
cameras causing a shut-down when saving RAW in continuous high-speed modes. Both
these problems were fixed so you can use dark-frame subtraction and RAW in
high-speed mode too. In earlier builds of CHDK we had to turn off Noise
Reduction if we wanted to use RAW in burst modes. As a result of these
experiments and old bugs in CHDK we got this fantastic Noise Reduction toggle.
When turned to ON it then applies a dark-frame routine at ALL shutter-speeds.
This removes all hot-pixels from all photos, no need to send your camera in to
have any hot, warm, and stuck pixels re-mapped. Setting it to Auto is just
leaving it at the camera's own defaults. Rightfully this "Noise Reduction"
option should be in the Misc. menu because it affects all JPGs too whether you
are saving RAW or not. Anyway, that's how it got where it presently sits in
CHDK's menus. Many people wonder why it was filed there since it's not just for
RAW anymore. Then too, because of this, they don't realize its other great
uses.)

Terry Randell wrote:

> Using this method you are able to get noise-free images at any ISO for any
> exposure length that you desire. Yes, I typed that correctly, at any ISO. Noise
> averages out and disappears when using frame-stacking methods with enough
> photos.
Noise averages out as the square root of the number of
images stacked up. You need 4 images to cut the noise in
half, 9 to cut the noise to a third, 16 to cut the noise to
a quarter, etc. It is a useful method, but it doesn't work
miracles. And if you use black frame subtraction, you have
to take into account the noise in the dark frame. These
methods are useful, but they don't work miracles.

And image artifacts from faulty pixels will add up nicely.

On Wed, 03 Oct 2007 16:13:24 GMT, Marvin <physchem@verizon.net> wrote:

>Terry Randell wrote:
>
>> Using this method you are able to get noise-free images at any ISO for any
>> exposure length that you desire. Yes, I typed that correctly, at any ISO. Noise
>> averages out and disappears when using frame-stacking methods with enough
>> photos.
>Noise averages out as the square root of the number of
>images stacked up. You need 4 images to cut the noise in
>half, 9 to cut the noise to a third, 16 to cut the noise to
>a quarter, etc. It is a useful method, but it doesn't work
>miracles. And if you use black frame subtraction, you have
>to take into account the noise in the dark frame. These
>methods are useful, but they don't work miracles.
>
>And image artifacts from faulty pixels will add up nicely.

Considering this is for a 30 minute exposure done with 15-second exposures, that
means there's 120 frames to stack. That's 1/11th the original noise. Since each
frame will have base noise removed with a dark-frame subtraction routine, and
since ISO80 in these cameras is already virtually zero noise you can't get much
better than that. If ISO800 is used and noise is reduced to 1/11th of that using
120 frames that means the noise is the same as if doing one image at the noise
level of using ISO72. Back to virtually zero noise, less than if one frame was
taken at ISO80, the camera's lowest ISO available, being near to zero noise.

I see no problem.

I don't think you understand the use of dark-frame subtraction. Any variance in
pixel values, the noise in the dark frame, is subtracted from the accompanying
images. Of course there's going to be noise in the dark-frame, that's the whole
purpose of doing this. Any noise that is not the base noise in the dark-frame
(pure sensor noise) will be averaged out from all 120 frames. Noise level = (see
above) virtually zero noise. What image artifacts from faulty pixels? That will
be registered as noise in the dark-frame noise subtraction. Perhaps you're not
familiar with the many methods of dark-frame subtraction. Download that
"BlackFrame NR" freeware and see the 4 different methods that can be used.
Stochastic Selective 2 Pass does some pretty nice things. This isn't just a
simple 3rd-graders' 137-12=125 subtraction for pixel values. Those "artifacts"
that you think will be there are averaged out from adjoining pixel data, no
different than having a sensor's pixels mapped out. In a friend's S3 IS sensor I
counted no more than 5 warm pixels up to a value of 14 (out of 255), and only 12
others giving a noise level of 2 to 7, the rest of the 5,947,375 pixels are all
at 0 or 1 noise reading in a 15 second exposure. (No dead or stuck pixels.) I
hardly see that as being a problem of lost data or the cause of obtrusive
artifacts.

With long-exposure sensor noise being this low in these cameras it's no wonder
that people have been clamoring to have extended shutter speeds. Canon really
dropped the ball on this one. They could have easily provided 1 to 4 minute
exposures on these cameras or even a bulb setting and nobody would have
complained in the least, while only giving them more praise. If this 30 minute
exposure is going to be for the night sky this isn't going to be putting
renegade asteroids or new stars in the sky. The worse that will happen is that a
star might be missed, but not likely since each star will be imaged by to 2 to 3
(or 4) pixels just due to atmospheric turbulence alone, so one non-working pixel
by any one star position won't matter. This will also provide crisper images
than a DSLR could accomplish because there's no camera shake being caused by the
built-in vibrations with each exposure from the mechanical linkage and moving
parts of that archaic focal-plane shutter (image jarring mirror locked or not),
thus smearing those point sources of light across adjoining pixels even further.

Getting close to virtually no noise at ISO800 with a 30 minute exposure out of
an inexpensive ($300 or less) P&S camera sounds pretty good to me. Not
miraculous you say? If people found a newly released $20,000 DSLR camera body,
not counting lens price, that could take a 30 minute exposure at ISO800 with
virtually zero noise I assure you that they'd be calling it the new miracle
camera. Only now that performance is already coming out of an inexpensive P&S
camera (one that was sold nearly 2 years ago, nearly 3 years ago if you use an
S2 IS for this). What would you call that if not "working miracles"?

On Thu, 04 Oct 2007 06:25:26 GMT, Terry Randell, aka Biddy, wrote:

> Getting close to virtually no noise at ISO800 with a 30 minute exposure out of
> an inexpensive ($300 or less) P&S camera sounds pretty good to me. Not
> miraculous you say? If people found a newly released $20,000 DSLR camera body,
> not counting lens price, that could take a 30 minute exposure at ISO800 with
> virtually zero noise I assure you that they'd be calling it the new miracle
> camera. Only now that performance is already coming out of an inexpensive P&S
> camera (one that was sold nearly 2 years ago, nearly 3 years ago if you use an
> S2 IS for this). What would you call that if not "working miracles"?

I'd call it another of your numerous trolls, sock puppet.