I'd like to learn Rust and thought it would be fun to procedurally generate images. I've no idea where to start though... piston/rust-image? But even with that where should I begin?
The place to begin is the docs and the repository.
It's not immediately obvious from the landing page of the documentation, but the core type in image is ImageBuffer.
The new function allows one to construct an ImageBuffer representing an image with the given/width, storing pixels of a given type (e.g. RGB, or that with transparency). One can use methods like pixels_mut, get_pixel_mut and put_pixel (the latter are below pixels_mut in the documentation) to modify the image. E.g.
extern crate image;
use image::{ImageBuffer, Rgb};
const WIDTH: u32 = 10;
const HEIGHT: u32 = 10;
fn main() {
// a default (black) image containing Rgb values
let mut image = ImageBuffer::<Rgb<u8>>::new(WIDTH, HEIGHT);
// set a central pixel to white
image.get_pixel_mut(5, 5).data = [255, 255, 255];
// write it out to a file
image.save("output.png").unwrap();
}
which looks like:
The repo is particularly useful as a starting point, because it contains examples, in particular, it has an example of programmatically generating an image. When using a new library, I'll open the docs, and, if confused, the repo specifically to look for examples.
NOTE:
get_pixel_mut Deprecated since 0.24.0: Use get_pixel and put_pixel instead.
As #huon answer is 6 years old, I was getting errors reproducing the result, so I wrote this,
use image::{ImageBuffer, RgbImage};
const WIDTH:u32 = 10;
const HEIGHT:u32 = 10;
fn main() {
let mut image: RgbImage = ImageBuffer::new(WIDTH, HEIGHT);
*image.get_pixel_mut(5, 5) = image::Rgb([255,255,255]);
image.save("output.png").unwrap();
}
Related
Cargo.toml
image = "0.23.12"
fltk = "0.10.14"
I'd like to save a rgb data as a jpeg file using rust's image crate:
use image::{RgbImage};
use image::ImageBuffer;
use fltk::{image::RgbImage, button::*};
let sourceImg = image::open("imgs/2.jpg").unwrap();
let rgb = RgbImage::new(&sourceImg.to_bytes(), x, y, 3).unwrap();
let mut prevWindowImgButton = Button::new(0,0, 300, 300, "");
prevWindowImgButton.set_image(Some(&rgb));
let rgbData= &prevWindowImgButton.image().unwrap().to_rgb_data();
//returns rgb data with type &Vec<u8>
rgbData.save("out/outputtest.jpg");
Gives error:
testRGB.save("out/outputtest.jpg");
| ^^^^ method not found in `&Vec<u8>`
Because .save must be used on an ImageBuffer. So how do I convert this rgb data to an ImageBuffer?
If all you want is to save a raw buffer to image file using the image crate you can use the save_buffer_with_format func:
use image::io::Reader;
use image::save_buffer_with_format;
fn main() {
// The following three lines simply load a test image and convert it into buffer
let img = Reader::open("myimage.png").unwrap().decode().unwrap().to_rgb8();
let (width, height) = (img.width(), img.height());
let img_byte_vec = img.into_raw();
// The next line is what you want
save_buffer_with_format("myimg.jpg", &img_byte_vec, width, height, image::ColorType::Rgb8, image::ImageFormat::Jpeg).unwrap();
}
Users write to screen and create image and send to our platform. Each image is a picture of something(hand written text) which is taken with a white background. I would like to remove all the white parts of the image get the content only user writes. Is it possible using image-processing in flutter? If possible how?
You can do image processing in Dart (Flutter is just a UI Framework, image processing is an algorithm, so not related to UI, but with the language itself).
You can try to use the image package, open the image, decode its bytes and apply any image processing algorithms there.
This library will help you extract the image bytes, but if you need more hardcore image processing algorithms, like the ones in OpenCV, and you aren't very keen on implementing them yourself, you might need to do some maneuvers like:
Test any of the OpenCV packages for Flutter/Dart in pub.dev. I can't recommend any as I never used them.
Use Method Channel calls to interact with OpenCV libraries for the native platforms (Android/iOS), passing the image and process it there and return it to Flutter afterwards.
Not ideal, but totally doable.
You can achieve same using remove.bg api. You can find more details on this thread.
First, you need to install Image package
Then use the following code to remove the background
import 'package:image/image.dart' as Img;
Future<Uint8List> removeWhiteBackgroundFromImage(
Uint8List bytes) async {
Img.Image image = Img.decodeImage(bytes);
Img.Image transparentImage = await _colorTransparent(image, 255, 255, 255);
var newPng = Img.encodePng(transparentImage);
return newPng;
}
Future<Img.Image> _colorTransparent(
Img.Image src, int red, int green, int blue) async {
var pixels = src.getBytes();
for (int i = 0, len = pixels.length; i < len; i += 4) {
if (pixels[i] == red && pixels[i + 1] == green && pixels[i + 2] == blue) {
pixels[i + 3] = 0;
}
}
return src;
}
Note: if you want to convert the image to Uint8List use the following code
Uint8List imageBytes = File(imageFilePath).readAsBytesSync();
Reference Link: https://gist.github.com/plateaukao/ebb5e7169dd89cc52bda338762d4997e
I'm using Piston's image crate, with this code:
use image::{Rgb, ImageBuffer, Pixel};
let image = Vec::<Rgb<u8>>::new();
let image_buffer = ImageBuffer::<Rgb<u8>, Vec<Rgb<u8>>>::from_vec(
width, height
image,
).unwrap();
However I get this error:
error[E0599]: no function or associated item named `from_vec` found for type `image::ImageBuffer<image::Rgb<u8>, std::vec::Vec<image::Rgb<u8>>>` in the current scope
--> src/main.rs:348:21
|
348 | let image_buffer = ImageBuffer::<Rgb<u8>, Vec<Rgb<u8>>>::from_vec(
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ function or associated item not found in `image::ImageBuffer<image::Rgb<u8>, std::vec::Vec<image::Rgb<u8>>>`
I can't work out why. It's clearly in the documentation, and the types seem right as far as I can tell.
Expanding a bit: In the example above, we have a ImageBuffer::<Rgb<u8>, Vec<Rgb<u8>>. And ImageBuffer provides two implementations of from_vec, depending on its type parameters:
impl<P, Container> ImageBuffer<P, Container>
where
P: Pixel<Subpixel = u8> + 'static,
Container: Deref<Target = [u8]>,
impl<P: Pixel + 'static> ImageBuffer<P, Vec<P::Subpixel>>
where
P::Subpixel: 'static,
Neither of these worked here because the Container parameter type in ImageBuffer<Rgb<u8>, Vec<Rgb<u8>> is a vector of Rgb<u8> values. It will dereference to a slice of [Rgb<u8>], making it incompatible with the first implementation, and the second one expects a vector of subpixel values (<P as Pixel>::Subpixel) rather than actual pixel values (Rgb<u8>). This is generally what the ImageBuffer type in this crate expects as its pixel data container.
Working example:
extern crate image;
use image::{ImageBuffer, Pixel, Rgb};
fn main() {
let width = 64;
let height = 64;
let image = vec![0x7F_u8; width as usize * height as usize * 3];
let image_buffer =
ImageBuffer::<Rgb<u8>, Vec<u8>>::from_vec(width, height, image).unwrap();
}
Playground
Ah it has to be a Vec<P::Subpixel>, i.e. Vec<u8> rather than a Vec<Rgb<u8>>. That is a little annoying.
I created an app in which I want to display text on top of google maps. I chose to use custom markers, but they can only be images, so I decided to create an image from my text utilizing SkiaSharp.
private static ImageSource CreateImageSource(string text)
{
int numberSize = 20;
int margin = 5;
SKBitmap bitmap = new SKBitmap(30, numberSize + margin * 2, SKImageInfo.PlatformColorType, SKAlphaType.Premul);
SKCanvas canvas = new SKCanvas(bitmap);
SKPaint paint = new SKPaint
{
Style = SKPaintStyle.StrokeAndFill,
TextSize = numberSize,
Color = SKColors.Red,
StrokeWidth = 1,
};
canvas.DrawText(text.ToString(), 0, numberSize, paint);
SKImage skImage = SKImage.FromBitmap(bitmap);
SKData data = skImage.Encode(SKEncodedImageFormat.Png, 100);
return ImageSource.FromStream(data.AsStream);
}
The images I create however have ugly artifacts on the top of the resulting image and my feeling is that they get worse if I create multiple images.
I built an example app, that shows the artifacts and the code I used to draw the text. It can be found here:
https://github.com/hot33331/SkiaSharpExample
How can I get rid of those artifacts. Am I using skia wrong?
I got the following answer from Matthew Leibowitz on the SkiaSharp GitHub:
The chances are you are not clearing the canvas/bitmap first.
You can either do bitmap.Erase(SKColors.Transparent) or canvas.Clear(SKColors.Transparent) (you can use any color).
The reason for this is performance. When creating a new bitmap, the computer has no way of knowing what background color you want. So, if it was to go transparent and you wanted white, then there would be two draw operations to clear the pixels (and this may be very expensive for large images).
During the allocation of the bitmap, the memory is provided, but the actual data is untouched. If there was anything there previously (which there will be), this data appears as colored pixels.
When I've seen that before, it's been because the memory passed to SkiaSharp was not zeroed. As an optimization, though, Skia assumes that the memory block passed to it is pre zeroed. Resultingly, if your first operation is a clear, it will ignore that operation, because it thinks that the state is already clean. To resolve this issue, you can manually zero the memory passed to SkiaSharp.
public static SKSurface CreateSurface(int width, int height)
{
// create a block of unmanaged native memory for use as the Skia bitmap buffer.
// unfortunately, this may not be zeroed in some circumstances.
IntPtr buff = System.Runtime.InteropServices.Marshal.AllocCoTaskMem(width * height * 4);
byte[] empty = new byte[width * height * 4];
// copy in zeroed memory.
// maybe there's a more sanctioned way to do this.
System.Runtime.InteropServices.Marshal.Copy(empty, 0, buff, width * height * 4);
// create the actual SkiaSharp surface.
var colorSpace = CGColorSpace.CreateDeviceRGB();
var bContext = new CGBitmapContext(buff, width, height, 8, width * 4, colorSpace, (CGImageAlphaInfo)bitmapInfo);
var surface = SKSurface.Create(width, height, SKColorType.Rgba8888, SKAlphaType.Premul, bitmap.Data, width * 4);
return surface;
}
Edit: btw, I assume this is a bug in SkiaSharp. The samples/apis that create the buffer for you should probably be zeroing it out. Depending on the platform it can be hard to repro as the memory alloc behaves differently. More or less likely to provide you untouched memory.
I am trying to track multiple objects with different color at the same time via a single web cam. Now I can do that for single color with single threshold:
IplImage* GetThresholdedImage(IplImage* imgHSV)
{
IplImage* imgThresh=cvCreateImage(cvGetSize(imgHSV),IPL_DEPTH_8U, 1);
cvInRangeS(imgHSV, cvScalar(170,160,60), cvScalar(180,2556,256), imgThresh);
return imgThresh;
}
I'm looking for some hints to do various threshold. Also if its possible, how many windows does it require? Do i need to assign different windows for different colors?
The easiest way to do this is create a thresholded image for each color you wish to track. Instead of hard-coding your threshold ranges, you could modify your function to take them as parameters. This lets you re-use the function for different objects.
The modified function might look like this:
IplImage* GetThresholdedImage(IplImage* imgHSV, CvScalar lower, CvScalar upper)
{
IplImage* imgThresh=cvCreateImage(cvGetSize(imgHSV),IPL_DEPTH_8U, 1);
cvInRangeS(imgHSV, lower, upper, imgThresh);
return imgThresh;
}
And then call it using different thresholds for different objects:
IplImage* hsv; /* Already initialized*/
/* Set thresholds for blue and green objects as an example. */
/*Obviously, set these to be whatever is necessary. */
CvScalar blue_lower = cvScalar(110,60,10);
CvScalar blue_upper = cvScalar(120,256,256);
CvScalar green_lower = cvScalar(40,60,10);
CvScalar green_upper = cvScalar(71,256,256);
/* Get the images thresholded for blue and green. */
IplImage* blue_mask = GetThresholdedImage(hsv, blue_lower, blue_upper);
IplImage* green_mask = GetThresholdedImage(hsv, green_lower, green_upper);