HDR merging
In this assignment, we will only work with images that are linear, static, and where the camera hasn't moved. The image files are provided with gamma 2.2 encoding and we have inserted lines of code in makeHDR and toneMap to decode and recode, respectively, the images. Keep these lines in your implementations of makeHDR and toneMap.
Consult the course slides for the overall merging approach. We will first compute weights for each pixel and each channel to eliminate values that are too high and too low. We will then compute the scale factor between the values of two images. Finally, we will merge a sequence of images using a weighted combination of the individual images according to the weights and scale factors.
Weights
FloatImage computeWeight( const FloatImage &im, float epsilonMini, float epsilonMaxi): that returns an image with pixel value 1.0 when the corresponding pixel value ofimis betweenepsilonMiniandepsilonMaxi, and 0.0 otherwise. Your output weight image should have the same dimensions as your input image im. Therefore, this operation should be done on a per pixel and per channel basis.
Use testComputeWeight in a5_main.cpp to help test this function.
Factor
Now that we know which pixels are usable in each picture, we can compute the multiplication factor between a pair of images.float computeFactor(const FloatImage &im1, const FloatImage &w1, const FloatImage &im2, const FloatImage &w2): that takes two images and their weights computed according to the above method, and returns a single scalar corresponding to the median value ofim2/im1for pixels that are usable in bothim1andim2. Add an epsilon of \(10^{-10}\) to the pixels ofim1to avoid divide by 0 errors.
Use testComputeFactor in a5_main.cpp to help test this function.
Merge to HDR
Use the previous functions to merge a sequence of images. You can assume that the first one is the darkest one, and that they are given in order of increasing exposure.FloatImage makeHDR(const vector: that takes a sequence of images as input and returns a single HDR image. Do not forget the special case for the computation of the weight of the darkest and brightest images. You should only threshold in one direction for these cases. If a pixel is not assigned to any of the weight images, then assign it the corresponding value from the first image in the sequance (by doing this you should avoid any divide by 0 errors as well). To test your method, you can write out the output image scaled by different factors.imageList, float epsilonMini, float epsilonMaxi)
Use testMakeHDR in a5_main.cpp to help test this function.
Tone mapping
Now that you have assembled HDR images, you are going to implement tone mapping. Your tone mapper will follow the method studied in class. The function will be called FloatImage toneMap(const Image &im, float targetBase, float detailAmp, bool useBila, float sigmaRange). It takes as input an HDR image, a target contrast for the base, an amplification factor for the detail, and a Boolean for the use of the bilateral filter vs. Gaussian blur.
As described in lecture slides, our goal is to reduce the contrast from the HDR image (say, 1:10000) to what we can show on a display (say 1:100). Although gamma correction would be the first thing we can think of, this results in washed out images, as you might have seen in class. The colors are actually okay (they're all there) but the high frequencies are washed out. We therefore want to work in the luminance, and increase the high frequencies, and we want to work in the log domain, since we are very sensitive to multiplicative contrast.
- First decompose your image into luminance and chrominance using the function from assignment 2:
vectorlumiChromi(const FloatImage &im) - Next, compute a log version of the luminance. Add a small constant (e.g. the minimum non-zero value) to the luminance to avoid problems at 0. We recommend you do this by first implementing the
float image_minnonzero(const FloatImage &im)function, followed byFloatImage log10Image(const FloatImage &im), for which we have written function signatures and comments for you. Then, calllog10Imagewith the luminance image as the argument.
Next, we want to extract the detail of the (log) luminance channel. We do this by blurring the log luminance, and subtracting the original. Convince yourself that gives you the details (high frequencies)!
- You are ready to compute the base (blurry) luminance. We will implement two versions: Gaussian blur and bilateral filtering, which will be chosen based on the value of the input parameter
useBila. In both cases, we will use a standard deviation for the spatial Gaussian equal to the biggest dimension of the image divided by 50. The parametertruncateDomainshould be set to the default value of 3. You are welcome to use your own implementation of filtering methods, but you can also use our versions in filtering.cpp. - Given the base, compute the detail by taking the difference from the original log luminance.
Great, you are finally ready to put everything together!
- Compute the scale factor
kin the log domain that brings the dynamic range of the base to the given target (that is, the range in the log domain should belog10(targetBase)after applyingk. We have provided two new functions for you that you might find useful:float min() constandfloat max() constin theFloatImageclass. Get a new base luminance using the factor, and multiply the details (in the log domain) bydetailAmp, then add the resulting output base and detail. Make sure to add an offset that ensures that the largest base value will be mapped to 1 once the image is put back into the linear domain. - Convert this new luminance back to the linear domain (you may want to separately implement
FloatImage exp10Image(const Image &im)). Then, reintegrate the chrominance into the resulting image. We've provided the functionFloatImage lumiChromi2rgb(const FloatImage &lumi, const Image &chromi)in basicImageManipulation.cpp which might be helpful.
Enjoy your results and compare the bilateral version with the Gaussian one. Use the functions testToneMapping_ante2, testToneMapping_ante3, testToneMapping_design, testToneMapping_boston in a5_main.cpp to help test your tone mapping function.
Your own HDR photo.
Take your own exposure bracketed sequence of images of a scene with high dynamic range. Think about what types of scenes can benefit from HDR imaging and tone mapping. Try to be creative. Stabilize your camera either on a tripod or by resting it firmly against a rigid object so that it moves minimally between shots (Make sure to disable any fancy instagram-like post-processing filters if you are doing this on a smartphone). At a bare minimum, your sequence should include two images. Ensure that the brightest exposure reveals details in the dark portions of the scene while having clipped whites, and your darkest exposure has no clipping of the brightest regions. Convert your images (most likely JPEGs) into PNGs, and include them in your submission.
Note: you can start portion 3.1 even before completing the coding tasks above.Use your code to merge your bracketed sequence of LDR images into an HDR image, and then tone map it for display. Explore different parameters for tone mapping, and choose the one you feel works best for your image. You can assume that your LDR images are gamma encoded with a gamma value of 2.2, and then linearize them before merging to HDR. Include your tone mapped result(s) in your submission.
Extra credit
Here are some ideas for extra credit, both for undergrads and grads (undergrads: up to 10%; grads: up to 5%).- (5%) Deal with image alignment (e.g. the sea images. We recommend Ward's median method (talk to me and I can point you in the right direction), but probably single-scale to make life easier yet slower. Alternatively, you can simulate the clipping in the darker of two images.
- (5%) Derive better weights by taking noise into account. You can focus on photon noise alone. This should give you an estimate of standard deviation (or something proportional to the standard deviation) for each pixel value in each image. Use a formula for the optimal combination as a function of variance to derive your weights while still properly avoiding clipped black and white pixels. Evaluate and compare with respect to the results using the naive binary weights.
- (5%) Write a function to calibrate the response curve of a camera. See http://www.pauldebevec.com/Research/HDR/
- (10%) Implement the bilaterial grid for fast bilateral filtering. See http://groups.csail.mit.edu/graphics/bilagrid/ with more mathematical justifications at http://people.csail.mit.edu/sparis/publi/2009/ijcv/Paris_09_Fast_Approximation.pdf
- (10%) Hard: deal with moving objects.
Submission
Turn in your files using Canvas and make sure all your files are in the a5 directory under the root of the zip file. Include all sources (.cpp and .h) files, any of your images, and the output of your program. Don't include your actual executable (we don't need your _build directory), and remove any superfluous files before submission.
In your readme.txt file, you should also answer the following questions:
- How long did the assignment take?
- Potential issues with your solution and explanation of partial completion (for partial credit)
- Any extra credit you may have implemented
- Collaboration acknowledgment (you must write your own code)
- What was most unclear