I am working on a software (Ruby - Sketchup) to calculate the radiation (sun, sky and surrounding buildings) within urban development at pedestrian level. The final goal is to be able to create a contour map that shows the level of total radiation. With total radiation I mean shortwave (light) and longwave(heat). (To give you an idea: http://www.iaacblog.com/maa2011-2012-digitaltools/files/2012/01/Insolation-Analysis-All-Year.jpg)
I know there are several existing software that do this, but I need to write my own as this calculation is only part of a more complex workflow.
The (obvious) pseudo code is the following:
Select and mesh surface for analysis
From each point of the mesh
Cast n (see below) rays in the upper hemisphere (precalculated)
For each ray check whether it is in shade
If in shade => Extract properties from intersected surface
If not in shade => Flag it
loop
loop
loop
The approach above is brute force, but it is the only I can think of. The calculation time increases with the fourth power of the accuracy (Dx,Dy,Dazimth, Dtilt). I know that software like radiance use a Montecarlo approach to reduce the number of rays.
As you can imagine, the accuracy of the calculation for a specific point of the mesh is strongly dependent by the accuracy of the skydome subdivision. Similarly the accuracy on the surface depends on the coarseness of the mesh.
I was thinking to a different approach using adaptive refinement based on the results of the calculations. The refinement could work for the surface analyzed and the skydome. If the results between two adjacent points differ more than a threshold value, than a refinement will be performed. This is usually done in fluid simulation, but I could not find anything about light simulation.
Also i wonder whether there are are algorithms, from computer graphics for example, that would allow to minimize the number of calculations. For example: check the maximum height of the surroundings so to exclude certain part of the skydome for certain points.
I don't need extreme accuracy as I am not doing rendering. My priority is speed at this moment.
Any suggestion on the approach?
Thanks
n rays
At the moment I subdivide the sky by constant azimuth and tilt steps; this causes irregular solid angles. There are other subdivisions (e.g. Tregenza) that maintain a constant solid angle.
EDIT: Response to the great questions from Spektre
Time frame. I run one simulation for each hour of the year. The weather data is extracted from an epw weather file. It contains, for each hour, solar altitude and azimuth, direct radiation, diffuse radiation, cloudiness (for atmospheric longwave diffuse). My algorithm calculates the shadow mask separately then it uses this shadow mask to calculate the radiation on the surface (and on a typical pedestrian) for each hour of the year. It is in this second step that I add the actual radiation. In the the first step I just gather information on the geometry and properties of the various surfaces.
Sun paths. No, i don't. See point 1
Include reflection from buildings? Not at the moment, but I plan to include it as an overall diffuse shortwave reflection based on sky view factor. I consider only shortwave reflection from the ground now.
Include heat dissipation from buildings? Absolutely yes. That is the reason why I wrote this code myself. Here in Dubai this is key as building surfaces gets very, very hot.
Surfaces albedo? Yes, I do. In Skethcup I have associated a dictionary to every surface and in this dictionary I include all the surface properties: temperature, emissivity, etc.. At the moment the temperatures are fixed (ambient temperature if not assigned), but I plan, in the future, to combine this with the results from a building dynamic thermal simulation that already calculates all the surfaces temperatures.
Map resolution. The resolution is chosen by the user and the mesh generated by the algorithm. In terms of scale, I use this for masterplans. The scale goes from 100mx100m up to 2000mx2000m. I usually tend to use a minimum resolution of 2m. The limit is the memory and the simulation time. I also have the option to refine specific areas with a much finer mesh: for example areas where there are restaurants or other amenities.
Framerate. I do not need to make an animation. Results are exported in a VTK file and visualized in Paraview and animated there just to show off during presentations :-)
Heat and light. Yes. Shortwave and longwave are handled separately. See point 4. The geolocalization is only used to select the correct weather file. I do not calculate all the radiation components. The weather files I need have measured data. They are not great, but good enough for now.
https://www.lucidchart.com/documents/view/5ca88b92-9a21-40a8-aa3a-0ff7a5968142/0
visible light
for relatively flat global base ground light map I would use projection shadow texture techniques instead of ray tracing angular integration. It is way faster with almost the same result. This will not work on non flat grounds (many bigger bumps which cast bigger shadows and also change active light absorbtion area to anisotropic). Urban areas are usually flat enough (inclination does not matter) so the technique is as follows:
camera and viewport
the ground map is a target screen so set the viewpoint to underground looking towards Sun direction upwards. Resolution is at least your map resolution and there is no perspective projection.
rendering light map 1st pass
first clear map with the full radiation (direct+diffuse) (light blue) then render buildings/objects but with diffuse radiation only (shadow). This will make the base map without reflections and or soft shadows in the Magenta rendering target
rendering light map 2nd pass
now you need to add building faces (walls) reflections for that I would take every outdoor face of the building facing Sun or heated enough and compute reflection points onto light map and render reflection directly to map
in tis parts you can add ray tracing for vertexes only to make it more precise and also for including multiple reflections (bu in that case do not forget to add scattering)
project target screen to destination radiation map
just project the Magenta rendering target image to ground plane (green). It is only simple linear affine transform ...
post processing
you can add soft shadows by blurring/smoothing the light map. To make it more precise you can add info to each pixel if it is shadow or wall. Actual walls are just pixels that are at 0m height above ground so you can use Z-buffer values directly for this. Level of blurring depends on the scattering properties of the air and of coarse pixels at 0m ground height are not blurred at all
IR
this can be done in similar way but temperature behaves a bit differently so I would make several layers of the scene in few altitudes above ground forming a volume render and then post process the energy transfers between pixels and layers. Also do not forget to add the cooling effect of green plants and water vaporisation.
I do not have enough experience in this field to make more suggestions I am more used to temperature maps for very high temperature variances in specific conditions and material not the outdoor conditions.
PS. I forgot albedo for IR and visible light is very different for many materials especially aluminium and some wall paintings
Related
I have problems with Z-fighting for my moving directional light casting shadows.
I'm trying to tinker with the rasterizer config when rendering the shadowmaps. There are three fields related to depth bias (DepthBias, SlopedScaleDepthBias, DepthBiasClamp) and no matter how I tinker with them I can't find a good value - I have no idea what value range is even appropriate.
Is there a universally-acceptable / "standard" set of sloped-scale/bias values that works for most scenes? I realize there are extreme cases but I just want the general case to work fine.
There isn't something like standard set of bias values. Implementing shadow maps is more like art than science :).
First ACME source is shadow map precision. Make sure that near and far planes of shadow casting light are as tight as possible. All objects before the near plane can be pancaked, as their exact depth isn't important.
Second ACME source is shadow map resolution. For a sunlight you should be doing cascaded shadow maps with at least 3 1024x1012 cascades for ~100-200m of shadow distance. It's hard to cover similar shadow distance with one shadow map with uniform projection.
Third ACME source are wide shadow map filters like PCF, as using a single depth depth comparison value across a wide kernel is insufficient. There are many methods to fix it, but none of them is robust.
To sum up, a tight frustum with a few cascades and some bias tweaking is enough to get the general case working. Start tweaking by disabling shadow filtering and tweak for filtering only when basic shadow maps are robust enough.
There are some additional interesting methods and most of them are implemented in the excellent demo: Matt Pettineo - "A sampling of shadow techniques".
Flip culling during shadow map rendering (this trades acne for Peter panning).
Normal offset shadow mapping does wonders for bias, but requires to have vertex normals around during shading.
Variance based methods (ESM,VSM,EVSM) completely remove bias issues, but have other drawbacks (light leaking and/or performance issues).
I am interested in using the Project Tango tablet for 3D reconstruction using arbitrary point features. In the current SDK version, we seem to have access to the following data.
A 1280 x 720 RGB image.
A point cloud with 0-~10,000 points, depending on the environment. This seems to average between 3,000 and 6,000 in most environments.
What I really want is to be able to identify a 3D point for key points within an image. Therefore, it makes sense to project depth into the image plane. I have done this, and I get something like this:
The problem with this process is that the depth points are sparse compared to the RGB pixels. So I took it a step further and performed interpolation between the depth points. First, I did Delaunay triangulation, and once I got a good triangulation, I interpolated between the 3 points on each facet and got a decent, fairly uniform depth image. Here are the zones where the interpolated depth is valid, imposed upon the RGB iamge.
Now, given the camera model, it's possible to project depth back into Cartesian coordinates at any point on the depth image (since the depth image was made such that each pixel corresponds to a point on the original RGB image, and we have the camera parameters of the RGB camera). However, if you look at the triangulation image and compare it to the original RGB image, you can see that depth is valid for all of the uninteresting points in the image: blank, featureless planes mostly. This isn't just true for this single set of images; it's a trend I'm seeing for the sensor. If a person stands in front of the sensor, for example, there are very few depth points within their silhouette.
As a result of this characteristic of the sensor, if I perform visual feature extraction on the image, most of the areas with corners or interesting textures fall in areas without associated depth information. Just an example: I detected 1000 SIFT keypoints from an an RGB image from an Xtion sensor, and 960 of those had valid depth values. If I do the same thing to this system, I get around 80 keypoints with valid depth. At the moment, this level of performance is unacceptable for my purposes.
I can guess at the underlying reasons for this: it seems like some sort of plane extraction algorithm is being used to get depth points, whereas Primesense/DepthSense sensors are using something more sophisticated.
So anyway, my main question here is: can we expect any improvement in the depth data at a later point in time, through improved RGB-IR image processing algorithms? Or is this an inherent limit of the current sensor?
I am from the Project Tango team at Google. I am sorry you are experiencing trouble with depth on the device. Just so that we are sure your device is in good working condition, can you please test the depth performance against a flat wall. Instructions are as below:
https://developers.google.com/project-tango/hardware/depth-test
Even with a device in good working condition, the depth library is known to return sparse depth points on scenes with low IR reflectance objects, small sized objects, high dynamic range scenes, surfaces at certain angles and objects at distances larger than ~4m. While some of these are inherent limitations in the depth solution, we are working with the depth solution provider to bring improvements wherever possible.
Attached an image of a typical conference room scene and the corresponding point cloud. As you can see, 1) no depth points are returned from the laptop screen (low reflectance), the table top objects such as post-its, pencil holder etc (small object sizes), large portions of the table (surface at an angles), room corner at the far right (distance >4m).
But as you move around the device, you will start getting depth point returns. Accumulating depth points is a must to get denser point clouds.
Please also keep us posted on your findings at project-tango-hardware-support#google.com
In my very basic initial experiments, you are correct with respect to depth information returned from the visual field, however, the return of surface points is anything but constant. I find as I move the device I can get major shifts in where depth information is returned, i.e. there's a lot of transitory opacity in the image with respect to depth data, probably due to the characteristics of the surfaces.
So while no return frame is enough, the real question seems to be the construction of a larger model (point cloud to open, possibly voxel spaces as one scales up) to bring successive scans into a common model. It's reminiscent of synthetic aperture algorithms in spirit, but the letters in the equations are from a whole different set of laws.
In short, I think a more interesting approach is to synthesize a more complete model by successive accumulation of point cloud data - now, for this to work, the device team has to have their dead reckoning on the money for whatever scale this is done. Also this addresses an issue that no sensor improvements can address - if your visual sensor is perfect, it still does nothing to help you relate the sides of an object at least be in the close neighborhood of the front of the object.
Many certain resources about raytracing tells about:
"shoot rays, find the first obstacle to cut it"
"shoot secondary rays..."
"or, do it reverse and approximate/interpolate"
I didnt see any algortihm that uses a diffusion algorithm. Lets assume a point-light is a point that has more density than other cells(all space is divided into cells), every step/iteration of lighting/tracing makes that source point to diffuse into neighbours using a velocity field and than their neighbours and continues like that. After some satisfactory iterations(such as 30-40 iterations), the density info of each cell is used for enlightment of objects in that cell.
Point light and velocity field:
But it has to be a like 1000x1000x1000 size and this would take too much time and memory to compute. Maybe just computing 10x10x10 and when finding an obstacle, partitioning that area to 100x100x100(in a dynamic kd-tree fashion) can help generating lighting/shadows for acceptable resolution? Especially for vertex-based illumination rather than triangle.
Has anyone tried this approach?
Note: Velocity field is here to make light diffuse to outwards mostly(not %100 but %99 to have some global illumination). Finite-element-method can make this embarassingly-parallel.
Edit: any object that is hit by a positive-density will be an obstacle to generate a new velocity field around the surface of it. So light cannot go through that object but can be mirrored to another direction.(if it is a lens object than light diffuse harder through it) So the reflection of light can affect other objects with a higher iteration limit
Same kd-tree can be used in object-collision algorithms :)
Just to take as a grain of salt: a neural-network can be trained for advection&diffusion in a 30x30x30 grid and that can be used in a "gpu(opencl/cuda)-->neural-network ---> finite element method --->shadows" way.
There's a couple problems with this as it stands.
The first problem is that, fundamentally, a photon in the Newtonian sense doesn't react or change based on the density of other photons around. So using a density field and trying to light to follow the classic Navier-Stokes style solutions (which is what you're trying to do, based on the density field explanation you gave) would result in incorrect results. It would also, given enough iterations, result in complete entropy over the scene, which is also not what happens to light.
Even if you were to get rid of the density problem, you're still left with the the problem of multiple photons going different directions in the same cell, which is required for global illumination and diffuse lighting.
So, stripping away the problem portions of your idea, what you're left with is a particle system for photons :P
Now, to be fair, sudo-particle systems are currently used for global illumination solutions. This type of thing is called Photon Mapping, but it's only simple to implement a direct lighting solution using it :P
We have multiple lights in 10x10 grid each of which we can control intensity 1 to 10. Target of those lights is a wall and our goal is to have uniform intensity within some range over wall image where user defines the intensity value. One restriction is that only direct adjacent neighbor lights of given light will be affect the image intensity for the wall area the light directly shed on.
I think (and hope) that this is a known problem but couldn't find any good reference to solve this problem. Any tip or clue would be appreciated.
I suppose that resulting intensity is linear combination of some neigbour lamps. For example, I[x,y]=a*L[x,y]+b*(L[x-1,y]+L[x+1,y]+L[x,y-1]+L[x,y-1])+c*(L[x-1,y-1] +...), where a,b,c are some coefficients. So there is linear system of 100 equations with 100 unknowns variables. It may be solved, if coefficients are known.
More complex model - convolution of lamp intensity matrix with point spread function. It may require sophisticated methods of signal reconstruction
This cries out for a genetic algorithms approach: Without too much trouble you can customize it to take into account your lamp characteristics, and any desired function of illumination on the wall.
Update: To be more concrete, if the OP already has some information about the light intensity function due to one lamp, then the programming aspect will be tedious, but straightforward. If not, then what's needed is a way to get that information. One way to do this is to get a photodiode and just measure the light intensity from the center to the periphery, with one lamp turned on mounted the way it will be in the real application. Use whatever sampling interval seems appropriate based on the physical set-up-- an inch, six inches, a foot, whatever. Using that information, the OP can create a function of light intensity based on one lamp.
I have no particular photodiode to recommend, but they can't be that expensive, since Lego Mindstorms can take readings from them. I did speak incorrectly in the comments below, though-- it might actually take one measurement for each of the ten intensity settings on the lamps, and I'm explicitly assuming that all the lamps have roughly the same performance.
From there, we can mathematically build the larger function of a light intensity pattern caused by 100 lamps at arbitrary intensities-- a function into which we can plug 100 numbers (representing the lamp settings) and get out a good approximation of the resulting light intensity. Finally, we can use a genetic algorithm to optimize the inputs of that function such that uniform intensity patterns are highly fit.
Careful, though-- the true optimum of that statement is probably "all lamps turned off."
(If you're more confident in your photography than I am, a camera might work. But either way, without a detailed knowledge of the intensity patterns of the lamp settings, this is not a solvable problem.)
I am working on an automatic image stitching algorithm using MATLAB. So far, I have downloaded a source code much like the one that I had in mind and so, I'm currently studying how the code work.
The problem is, when stitching two or more images together, their color intensity will most probably be different from each other so the stitched seams will be visible to the eye... So, right now, I'm trying to find out how to redistribute their color intensity using the images gradients so that the whole stitched image will have the same color intensity.
I hope someone can help me out there and if so, thank you very much...
If the images overlap by a significant amount, and the stitching algorithm does a very good job of registering the overlap region, a very simple solution would be to blend the pixel values from the two images together in the overlap region, using a weighted average with weights going from 0-1 depending on the distance from the edge of the overlap region.
blendedPixel = (imageApixel * weightA) + (imageBpixel * weightB)
where weightA is approaches 1 as we get closer to the imageA side of the overlap region, weightB approaches 1 as we get closer to the imageB side of the overlap region, and the sum of weightA and weightB is always 1.
The above solution is not particularly principled, and does depend on the stitching algorithm doing a very good job of image registration in the overlap region.
Another, more principled solution to the problem would be to remove the source of the intensity difference, attempting to homogenize the response of the pixels across the image plane.
The form of this solution will depend on the source of the intensity difference, which will depend on the optics and the scene lighting conditions.
For example when dealing with photographs of outdoor scenes, taken at the same time from the same location, then the dominant effect will likely be "vignetting" effects, which can be due to a variety of different causes, including differences between the various paths taken by the light through camera optics.
As another example, when dealing with photographs taken through a microscope of a sample illuminated at an oblique angle, the dominant effect will likely be due to the difference in illumination between those parts of the image closest to the light and those far away.
Vignetting generally manifests itself as a radially symmetric function centred around the projection of the optical axis of the lens onto the image plane. To correct for vignetting, you should try to fit a suitable radially symmetric function.
Lighting changes can take different functional forms, but fitting a straightforward linear approximation is sufficient in many cases.
Depending upon the scene, and the number and variability of the images that have available, you may need to take calibration images to fit these functions properly.
The above approaches make assumptions about the functional forms of the sources of the intensity differences, but not about the scene or it's statistics.
Yet another approach might be to make some assumptions about the scene, for example, that all significant information is represented by spatial frequencies above some threshold. You can then remove all low image intensity spatial frequency components. This will "flatten" the image, removing much of the low-frequency vignetting and lighting issues.
This approach might be applicable to microscopy images, sattelite images, or images of other scenes within which most of the interest lies in the detail, rather than in the drama of the composition.
There are a number of papers that tackle this problem, many at a level of technical sophistication rather beyond the above discussion. For example, see D Goldman, "Vignette and Exposure Calibration and Compensation", IEEE Trans Pattern Analysis and Machine Intelligence, vol 32, no 12, pp2276-2288