Swift 3: get color of pixel in UIImage (better: UIImageView) - uiimageview

I tried different solutions (e.g. this one), but the color I get back looks a bit different than in the real image. I guess it's because the image is only RGB, not RGBA. May that be an issue?
Related issue: if the UIImage has contentMode = .scaleAspectFill, do I have to do a recalculation of the image or can I just use imageView.image?
EDIT:
I tried with this extension:
extension CALayer {
func getPixelColor(point: CGPoint) -> CGColor {
var pixel: [CUnsignedChar] = [0, 0, 0, 0]
let colorSpace = CGColorSpaceCreateDeviceRGB()
let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedLast.rawValue)
let context = CGContext(data: &pixel, width: 1, height: 1, bitsPerComponent: 8, bytesPerRow: 4, space: colorSpace, bitmapInfo: bitmapInfo.rawValue)
context!.translateBy(x: -point.x, y: -point.y)
self.render(in: context!)
let red: CGFloat = CGFloat(pixel[0]) / 255.0
let green: CGFloat = CGFloat(pixel[1]) / 255.0
let blue: CGFloat = CGFloat(pixel[2]) / 255.0
let alpha: CGFloat = CGFloat(pixel[3]) / 255.0
let color = UIColor(red:red, green: green, blue:blue, alpha:alpha)
return color.cgColor
}
}
but for some images it seems as if the coordinate system is turned around, for others I get really wrong values... what am I missing here?
EDIT 2:
I try with these images:
https://dl.dropboxusercontent.com/u/119600/gradient.png
https://dl.dropboxusercontent.com/u/119600/gradient#2x.png
but I do get wrong values. They are embedded in a UIImageView but I convert the coordinates:
private func convertScreenPointToImage(point: CGPoint) -> CGPoint {
let widthMultiplier = gradientImage.size.width / UIScreen.main.bounds.width
let heightMultiplier = gradientImage.size.height / UIScreen.main.bounds.height
return CGPoint(x: point.x * widthMultiplier, y: point.y * heightMultiplier)
}
This one
gives me === Optional((51, 76, 184, 255)) when running on the iPhone 7 simulator, which is not correct...

I wrote this is a playground. I index into the image data with a pointer and grab the rgba values:
func pixel(in image: UIImage, at point: CGPoint) -> (UInt8, UInt8, UInt8, UInt8)? {
let width = Int(image.size.width)
let height = Int(image.size.height)
let x = Int(point.x)
let y = Int(point.y)
guard x < width && y < height else {
return nil
}
guard let cfData:CFData = image.cgImage?.dataProvider?.data, let pointer = CFDataGetBytePtr(cfData) else {
return nil
}
let bytesPerPixel = 4
let offset = (x + y * width) * bytesPerPixel
return (pointer[offset], pointer[offset + 1], pointer[offset + 2], pointer[offset + 3])
}
let image = UIImage(named: "t.png")!
if let (r,g,b,a) = pixel(in: image, at: CGPoint(x: 1, y:2)) {
print ("Red: \(r), Green: \(g), Blue: \(b), Alpha: \(a)")
}
Note that if you use this on a UIImage that is a property of a UIImageView the pixel coordinates are those of the actual image in its original resolution, not the screen coordinates of the scaled UIImageView. Also it tried with RGB Jpg and RGBA PNG and the both get imported as 32 bit RGBA images so it works for both.

Related

How to interpret result of MLMultiArray in semantic segmentation through CoreML?

I am trying to implement a semantic segmentation model in my application. I have been able to convert the u2net model to a CoreML model. I am unable to get a workable result from the MLMultiArray output. The specification description is as follows:
input {
name: "input"
type {
imageType {
width: 512
height: 512
colorSpace: RGB
}
}
}
output {
name: "x_1"
type {
multiArrayType {
shape: 1
shape: 3
shape: 512
shape: 512
dataType: FLOAT32
}
}
}
The model works great when opening it and using the model preview functionality in Xcode. It shows the 2 different labels in 2 colours (there are only 2 classes + 1 background). I want to have the same output in my application, however when I manually process the MLMultiArray output to a CGImage I get different results. I am using the code provided here like this:
let image = output.cgImage(min: -1, max: 1, channel: 0, axes: (1,2,3))
This gives me something that looks somewhat usable but it has a lot of gradient within each channel. What I need is an image with simply 1 color value for each label.
I have tried converting the output of the model directly to an image through this sample code. This simply shows 'Inference Failed' in the Xcode model preview. When I try removing the unnecessary extra dimension in the MultiArray output I get this error:
"Error reading protobuf spec. validator error: Layer 'x_1' of type 'Convolution' has output rank 3 but expects rank at least 4."
What does the model preview in Xcode do what I am not doing? Is there a post processing step I need to take to get usable output?
Answering my own question:
Turns out the resulting pixels for each channel represent the possibility of it being the class represented by that channel.
In other words find the maximum pixel value at a certain position. The channel with the highest value is the class pixel.
func getLabelsForImage() {
....
setup model here
....
guard let output = try? model.prediction(input: input) else {
fatalError("Could not generate model output.")
}
let channelCount = 10
// Ugly, I know. But works:
let colors = [NSColor.red.usingColorSpace(.sRGB)!, NSColor.blue.usingColorSpace(.sRGB)!, NSColor.green.usingColorSpace(.sRGB)!, NSColor.gray.usingColorSpace(.sRGB)!, NSColor.yellow.usingColorSpace(.sRGB)!, NSColor.purple.usingColorSpace(.sRGB)!, NSColor.cyan.usingColorSpace(.sRGB)!, NSColor.orange.usingColorSpace(.sRGB)!, NSColor.brown.usingColorSpace(.sRGB)!, NSColor.magenta.usingColorSpace(.sRGB)!]
// I don't know my min and max output, -64 and 64 seems to work OK for my data.
var firstData = output.toRawBytes(min: Float32(-64), max: Float32(64), channel: 0, axes: (0,1,2))!.bytes
var outputImageData:[UInt8] = []
for _ in 0..<firstData.count {
let r:UInt8 = UInt8(colors[0].redComponent * 255)
let g:UInt8 = UInt8(colors[0].greenComponent * 255)
let b:UInt8 = UInt8(colors[0].blueComponent * 255)
let a:UInt8 = UInt8(colors[0].alphaComponent * 255)
outputImageData.append(r)
outputImageData.append(g)
outputImageData.append(b)
outputImageData.append(a)
}
for i in 1..<channelCount {
let data = output.toRawBytes(min: Float32(-64), max: Float32(64), channel: i, axes: (0,1,2))!.bytes
for j in 0..<data.count {
if data[j] > firstData[j] {
firstData[j] = data[j]
let r:UInt8 = UInt8(colors[i].redComponent * 255)
let g:UInt8 = UInt8(colors[i].greenComponent * 255)
let b:UInt8 = UInt8(colors[i].blueComponent * 255)
let a:UInt8 = UInt8(colors[i].alphaComponent * 255)
outputImageData[j*4] = r
outputImageData[j*4+1] = g
outputImageData[j*4+2] = b
outputImageData[j*4+3] = a
}
}
}
let image = imageFromPixels(pixels: outputImageData, width: 512, height: 512)
image.writeJPG(toURL: labelURL.deletingLastPathComponent().appendingPathComponent("labels.jpg"))
}
// I found this function here: https://stackoverflow.com/questions/38590323/obtain-nsimage-from-pixel-array-problems-swift
func imageFromPixels(pixels: UnsafePointer<UInt8>, width: Int, height: Int)-> NSImage { //No need to pass another CGImage
let rgbColorSpace = CGColorSpaceCreateDeviceRGB()
let bitmapInfo:CGBitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.premultipliedLast.rawValue)
let bitsPerComponent = 8 //number of bits in UInt8
let bitsPerPixel = 4 * bitsPerComponent //ARGB uses 4 components
let bytesPerRow = bitsPerPixel * width / 8 // bitsPerRow / 8 (in some cases, you need some paddings)
let providerRef = CGDataProvider(
data: NSData(bytes: pixels, length: height * bytesPerRow) //Do not put `&` as pixels is already an `UnsafePointer`
)
let cgim = CGImage(
width: width,
height: height,
bitsPerComponent: bitsPerComponent,
bitsPerPixel: bitsPerPixel,
bytesPerRow: bytesPerRow, //->not bits
space: rgbColorSpace,
bitmapInfo: bitmapInfo,
provider: providerRef!,
decode: nil,
shouldInterpolate: true,
intent: CGColorRenderingIntent.defaultIntent
)
return NSImage(cgImage: cgim!, size: NSSize(width: width, height: height))
}

How to Animate a UIImage out of the Screen in Swift

i have searched around the web but nothing could answer my question.
I am trying to animate an UIImageview out of the Screen, set the alpha of it to 0, reset its position and reset the alpha to 1 again.
I can't seem to get the right position where to animate the Image to.
I have set up a gesturerecognizer to drag the image around and when the center of the image is above a certain point, i want to animate it out of screen.
This is the Regocnizercode :
#objc func imageDragged(gestureRecognizer: UIPanGestureRecognizer){
//Current Point where the label is dragged
let draggedLabelPoint = gestureRecognizer.translation(in: view)
//Updating the center of the label : viewcenter +/- currentpoint
matchImageView.center = CGPoint(x: view.bounds.width/2 + draggedLabelPoint.x, y: view.bounds.height/2 + draggedLabelPoint.y)
let xFromCenter = view.bounds.width/2 - matchImageView.center.x
var rotation = CGAffineTransform(rotationAngle: xFromCenter / -500)
let scale = min(100/abs(xFromCenter),1)
var scaledAndRotated = rotation.scaledBy(x: scale, y: scale)
//Transforming the Item respective to the distance from center
matchImageView.transform = scaledAndRotated
if gestureRecognizer.state == .ended {
if matchImageView.center.x < (view.bounds.width/2 - 100) {
animateImageToSide(target: CGPoint(x: view.bounds.width - 200, y: matchImageView.center.y))
matchImageView.alpha = 0
}
if matchImageView.center.x > (view.bounds.width/2 + 100) {
animateImageToSide(target: CGPoint(x: view.bounds.width + 200, y: matchImageView.center.y))
matchImageView.alpha = 0
}
//Reset the scaling variables and recentering the Item after swipe
rotation = CGAffineTransform(rotationAngle: 0)
scaledAndRotated = rotation.scaledBy(x: 1, y: 1)
matchImageView.transform = scaledAndRotated
matchImageView.center = CGPoint(x: view.bounds.width/2, y: view.bounds.height/2)
}
}
I am sorry if the resetting part is irrelevant for you, but im not sure if this could cause this behavior.
And this is my current Animationmethod :
func animateImageToSide(target:CGPoint){
UIImageView.animate(withDuration: 0.5, delay: 0, options: .curveLinear, animations: {self.matchImageView.center = target}) { (success: Bool) in
self.updateImage()
self.fadeInImage()//This is where i set the alpha back to 1
}
}
I am not sure if a CGPoint is what i need to use.
The current behavoir of the image is very weird. Most of the time it snaps to a specific place, regardless of the targetposition. Maybe my attempt on this is entirely wrong.
Thanks for your help !

Draw a vertical line

I want to put an image (I will create) in my status bar app and this image should made of vertical line but I don't now how to draw them.
Found
let graphWidth: CGFloat = 22.0;
let graphHeight: CGFloat = 22.0
let size = NSMakeSize( graphWidth, graphHeight )
let image = NSImage(size: size)
image.lockFocus()
let path = NSBezierPath()
path.move(to: NSPoint(x: 20.0, y: 0))
path.line(to: NSPoint(x: 20.0, y: 20.0))
path.stroke()
image.unlockFocus()
statusItem.image = image

Getting Pixel Color from an Image using CGPoint in Swift 3

I am try this PixelExtractor class in Swift 3, get a error;
Cannot invoke initializer for type 'UnsafePointer' with an argument list of type '(UnsafeMutableRawPointer?)'
class PixelExtractor: NSObject {
let image: CGImage
let context: CGContextRef?
var width: Int {
get {
return CGImageGetWidth(image)
}
}
var height: Int {
get {
return CGImageGetHeight(image)
}
}
init(img: CGImage) {
image = img
context = PixelExtractor.createBitmapContext(img)
}
class func createBitmapContext(img: CGImage) -> CGContextRef {
// Get image width, height
let pixelsWide = CGImageGetWidth(img)
let pixelsHigh = CGImageGetHeight(img)
let bitmapBytesPerRow = pixelsWide * 4
let bitmapByteCount = bitmapBytesPerRow * Int(pixelsHigh)
// Use the generic RGB color space.
let colorSpace = CGColorSpaceCreateDeviceRGB()
// Allocate memory for image data. This is the destination in memory
// where any drawing to the bitmap context will be rendered.
let bitmapData = malloc(bitmapByteCount)
let bitmapInfo = CGBitmapInfo(rawValue: CGImageAlphaInfo.PremultipliedFirst.rawValue)
let size = CGSizeMake(CGFloat(pixelsWide), CGFloat(pixelsHigh))
UIGraphicsBeginImageContextWithOptions(size, false, 0.0)
// create bitmap
let context = CGBitmapContextCreate(bitmapData, pixelsWide, pixelsHigh, 8,
bitmapBytesPerRow, colorSpace, bitmapInfo.rawValue)
// draw the image onto the context
let rect = CGRect(x: 0, y: 0, width: pixelsWide, height: pixelsHigh)
CGContextDrawImage(context, rect, img)
return context!
}
func colorAt(x x: Int, y: Int)->UIColor {
assert(0<=x && x<width)
assert(0<=y && y<height)
let uncastedData = CGBitmapContextGetData(context)
let data = UnsafePointer<UInt8>(uncastedData)
let offset = 4 * (y * width + x)
let alpha: UInt8 = data[offset]
let red: UInt8 = data[offset+1]
let green: UInt8 = data[offset+2]
let blue: UInt8 = data[offset+3]
let color = UIColor(red: CGFloat(red)/255.0, green: CGFloat(green)/255.0, blue: CGFloat(blue)/255.0, alpha: CGFloat(alpha)/255.0)
return color
}
}
Fix this error.
let data = UnsafePointer<UInt8>(uncastedData)
->
let data = UnsafeRawPointer(uncastedData)
Get other error; 'Type 'UnsafeRawPointer?' has no subscript members'
How to modify this error?
You can write something like this when you have an UnsafeRawPointer in your data:
let alpha = data.load(fromByteOffset: offset, as: UInt8.self)
let red = data.load(fromByteOffset: offset+1, as: UInt8.self)
let green = data.load(fromByteOffset: offset+2, as: UInt8.self)
let blue = data.load(fromByteOffset: offset+3, as: UInt8.self)
Or else, you can get UnsafeMutablePointer<UInt8> from your uncastedData (assuming it's an UnsafeMutableRawPointer):
let data = uncastedData.assumingMemoryBound(to: UInt8.self)
SWIFT 3 (updated March 2017) Xcode 8 / IOS 10
Important: note that return value corresponds to red: b, green:r and blue: r as in the data they are stored backwards
First, create the extension (you can copy&paste somewhere in your
code)
extension UIImage {
func getPixelColor(pos: CGPoint) -> UIColor {
if let pixelData = self.cgImage?.dataProvider?.data {
let data: UnsafePointer<UInt8> = CFDataGetBytePtr(pixelData)
let pixelInfo: Int = ((Int(self.size.width) * Int(pos.y)) + Int(pos.x)) * 4
let r = CGFloat(data[pixelInfo+0]) / CGFloat(255.0)
let g = CGFloat(data[pixelInfo+1]) / CGFloat(255.0)
let b = CGFloat(data[pixelInfo+2]) / CGFloat(255.0)
let a = CGFloat(data[pixelInfo+3]) / CGFloat(255.0)
return UIColor(red: b, green: g, blue: r, alpha: a)
} else {
//IF something is wrong I returned WHITE, but change as needed
return UIColor.white
}
}
}
Then just call it as:
let colorAtPixel : UIColor = (theView.image?.getPixelColor(pos: CGPoint(x: 2, y: 2)))!
Although the code returns de exact color, it seems that is not returning the correct one for different CGPoints.
Might it be because of the screen resolution? (x1,x2,x3)?
It would be great if someone can add some light to the mystery...
Swift-3 (IOS 10.3)
extension UIImage {
func getPixelColor(atLocation location: CGPoint, withFrameSize size: CGSize) -> UIColor {
let x: CGFloat = (self.size.width) * location.x / size.width
let y: CGFloat = (self.size.height) * location.y / size.height
let pixelPoint: CGPoint = CGPoint(x: x, y: y)
let pixelData = self.cgImage!.dataProvider!.data
let data: UnsafePointer<UInt8> = CFDataGetBytePtr(pixelData)
let pixelIndex: Int = ((Int(self.size.width) * Int(pixelPoint.y)) + Int(pixelPoint.x)) * 4
let r = CGFloat(data[pixelIndex]) / CGFloat(255.0)
let g = CGFloat(data[pixelIndex+1]) / CGFloat(255.0)
let b = CGFloat(data[pixelIndex+2]) / CGFloat(255.0)
let a = CGFloat(data[pixelIndex+3]) / CGFloat(255.0)
return UIColor(red: r, green: g, blue: b, alpha: a)
}
}
Usage : -
let color = yourImageView.image!.getPixelColor(atLocation: location, withFrameSize: yourImageView.frame.size)
location is a CGPoint
and size is size of your imageView
The following section is taken from some Swift 3 code I'm using to sample pixels from an image to get the predominant hue which I use to generate a background for tableView rows. The mechanics for the hue selection process don't apply to your question, so I'm just providing the relevant fragment.
let colorSpace = CGColorSpaceCreateDeviceRGB() // UIExtendedSRGBColorSpace
let newImage = image.cgImage?.copy(colorSpace: colorSpace)
let pixelData = newImage?.dataProvider!.data
let data: UnsafePointer<UInt8> = CFDataGetBytePtr(pixelData)
var hueFrequency = [Int: Double]()
hueFrequency[1] = 1 // Add one entry so this serves as a default if no hues from the image pass the filters
let nStart = 1
let mStart = 1
for n in nStart...Int(image.size.width / samplingFactor) {
for m in mStart...Int(image.size.height / samplingFactor) {
let pixelInfo: Int = ((Int(image.size.width) * m * Int(samplingFactor)) + n * Int(samplingFactor)) * 4 // bytesPerPixel
let b = CGFloat(data[pixelInfo]) / CGFloat(255.0) // cgImage bitmapinfo = rawValue 8194 -> BGRA ordering
let g = CGFloat(data[pixelInfo+1]) / CGFloat(255.0)
let r = CGFloat(data[pixelInfo+2]) / CGFloat(255.0)
let a = CGFloat(data[pixelInfo+3]) / CGFloat(255.0)
Also, note that I found the bitmapInfo value (image.cgImage!.bitmapInfo using my parameters) indicated a reordering of the RGBA sequence to BGRA, which I had to deal with in ordering the steps to pick out the data. If your colors are off, you may want to check this.

How to get pixel data from a UIImage (Cocoa Touch) or CGImage (Core Graphics)?

I have a UIImage (Cocoa Touch). From that, I'm happy to get a CGImage or anything else you'd like that's available. I'd like to write this function:
- (int)getRGBAFromImage:(UIImage *)image atX:(int)xx andY:(int)yy {
// [...]
// What do I want to read about to help
// me fill in this bit, here?
// [...]
int result = (red << 24) | (green << 16) | (blue << 8) | alpha;
return result;
}
FYI, I combined Keremk's answer with my original outline, cleaned-up the typos, generalized it to return an array of colors and got the whole thing to compile. Here is the result:
+ (NSArray*)getRGBAsFromImage:(UIImage*)image atX:(int)x andY:(int)y count:(int)count
{
NSMutableArray *result = [NSMutableArray arrayWithCapacity:count];
// First get the image into your data buffer
CGImageRef imageRef = [image CGImage];
NSUInteger width = CGImageGetWidth(imageRef);
NSUInteger height = CGImageGetHeight(imageRef);
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
unsigned char *rawData = (unsigned char*) calloc(height * width * 4, sizeof(unsigned char));
NSUInteger bytesPerPixel = 4;
NSUInteger bytesPerRow = bytesPerPixel * width;
NSUInteger bitsPerComponent = 8;
CGContextRef context = CGBitmapContextCreate(rawData, width, height,
bitsPerComponent, bytesPerRow, colorSpace,
kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
CGColorSpaceRelease(colorSpace);
CGContextDrawImage(context, CGRectMake(0, 0, width, height), imageRef);
CGContextRelease(context);
// Now your rawData contains the image data in the RGBA8888 pixel format.
NSUInteger byteIndex = (bytesPerRow * y) + x * bytesPerPixel;
for (int i = 0 ; i < count ; ++i)
{
CGFloat alpha = ((CGFloat) rawData[byteIndex + 3] ) / 255.0f;
CGFloat red = ((CGFloat) rawData[byteIndex] ) / alpha;
CGFloat green = ((CGFloat) rawData[byteIndex + 1] ) / alpha;
CGFloat blue = ((CGFloat) rawData[byteIndex + 2] ) / alpha;
byteIndex += bytesPerPixel;
UIColor *acolor = [UIColor colorWithRed:red green:green blue:blue alpha:alpha];
[result addObject:acolor];
}
free(rawData);
return result;
}
One way of doing it is to draw the image to a bitmap context that is backed by a given buffer for a given colorspace (in this case it is RGB): (note that this will copy the image data to that buffer, so you do want to cache it instead of doing this operation every time you need to get pixel values)
See below as a sample:
// First get the image into your data buffer
CGImageRef image = [myUIImage CGImage];
NSUInteger width = CGImageGetWidth(image);
NSUInteger height = CGImageGetHeight(image);
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
unsigned char *rawData = malloc(height * width * 4);
NSUInteger bytesPerPixel = 4;
NSUInteger bytesPerRow = bytesPerPixel * width;
NSUInteger bitsPerComponent = 8;
CGContextRef context = CGBitmapContextCreate(rawData, width, height, bitsPerComponent, bytesPerRow, colorSpace, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
CGColorSpaceRelease(colorSpace);
CGContextDrawImage(context, CGRectMake(0, 0, width, height));
CGContextRelease(context);
// Now your rawData contains the image data in the RGBA8888 pixel format.
int byteIndex = (bytesPerRow * yy) + xx * bytesPerPixel;
red = rawData[byteIndex];
green = rawData[byteIndex + 1];
blue = rawData[byteIndex + 2];
alpha = rawData[byteIndex + 3];
Apple's Technical Q&A QA1509 shows the following simple approach:
CFDataRef CopyImagePixels(CGImageRef inImage)
{
return CGDataProviderCopyData(CGImageGetDataProvider(inImage));
}
Use CFDataGetBytePtr to get to the actual bytes (and various CGImageGet* methods to understand how to interpret them).
Couldn't believe that there is not one single correct answer here. No need to allocate pointers, and the unmultiplied values still need to be normalized. To cut to the chase, here is the correct version for Swift 4. For UIImage just use .cgImage.
extension CGImage {
func colors(at: [CGPoint]) -> [UIColor]? {
let colorSpace = CGColorSpaceCreateDeviceRGB()
let bytesPerPixel = 4
let bytesPerRow = bytesPerPixel * width
let bitsPerComponent = 8
let bitmapInfo: UInt32 = CGImageAlphaInfo.premultipliedLast.rawValue | CGBitmapInfo.byteOrder32Big.rawValue
guard let context = CGContext(data: nil, width: width, height: height, bitsPerComponent: bitsPerComponent, bytesPerRow: bytesPerRow, space: colorSpace, bitmapInfo: bitmapInfo),
let ptr = context.data?.assumingMemoryBound(to: UInt8.self) else {
return nil
}
context.draw(self, in: CGRect(x: 0, y: 0, width: width, height: height))
return at.map { p in
let i = bytesPerRow * Int(p.y) + bytesPerPixel * Int(p.x)
let a = CGFloat(ptr[i + 3]) / 255.0
let r = (CGFloat(ptr[i]) / a) / 255.0
let g = (CGFloat(ptr[i + 1]) / a) / 255.0
let b = (CGFloat(ptr[i + 2]) / a) / 255.0
return UIColor(red: r, green: g, blue: b, alpha: a)
}
}
}
The reason you have to draw/convert the image first into a buffer is because images can have several different formats. This step is required to convert it to a consistent format you can read.
NSString * path = [[NSBundle mainBundle] pathForResource:#"filename" ofType:#"jpg"];
UIImage * img = [[UIImage alloc]initWithContentsOfFile:path];
CGImageRef image = [img CGImage];
CFDataRef data = CGDataProviderCopyData(CGImageGetDataProvider(image));
const unsigned char * buffer = CFDataGetBytePtr(data);
Here is a SO thread where #Matt renders only the desired pixel into a 1x1 context by displacing the image so that the desired pixel aligns with the one pixel in the context.
Swift 5 version
The answers given here are either outdated or incorrect because they don't take into account the following:
The pixel size of the image can differ from its point size that is returned by image.size.width/image.size.height.
There can be various layouts used by pixel components in the image, such as BGRA, ABGR, ARGB etc. or may not have an alpha component at all, such as BGR and RGB. For example, UIView.drawHierarchy(in:afterScreenUpdates:) method can produce BGRA images.
Color components can be premultiplied by the alpha for all pixels in the image and need to be divided by alpha in order to restore the original color.
For memory optimization used by CGImage, the size of a pixel row in bytes can be greater than the mere multiplication of the pixel width by 4.
The code below is to provide a universal Swift 5 solution to get the UIColor of a pixel for all such special cases. The code is optimized for usability and clarity, not for performance.
public extension UIImage {
var pixelWidth: Int {
return cgImage?.width ?? 0
}
var pixelHeight: Int {
return cgImage?.height ?? 0
}
func pixelColor(x: Int, y: Int) -> UIColor {
assert(
0..<pixelWidth ~= x && 0..<pixelHeight ~= y,
"Pixel coordinates are out of bounds")
guard
let cgImage = cgImage,
let data = cgImage.dataProvider?.data,
let dataPtr = CFDataGetBytePtr(data),
let colorSpaceModel = cgImage.colorSpace?.model,
let componentLayout = cgImage.bitmapInfo.componentLayout
else {
assertionFailure("Could not get a pixel of an image")
return .clear
}
assert(
colorSpaceModel == .rgb,
"The only supported color space model is RGB")
assert(
cgImage.bitsPerPixel == 32 || cgImage.bitsPerPixel == 24,
"A pixel is expected to be either 4 or 3 bytes in size")
let bytesPerRow = cgImage.bytesPerRow
let bytesPerPixel = cgImage.bitsPerPixel/8
let pixelOffset = y*bytesPerRow + x*bytesPerPixel
if componentLayout.count == 4 {
let components = (
dataPtr[pixelOffset + 0],
dataPtr[pixelOffset + 1],
dataPtr[pixelOffset + 2],
dataPtr[pixelOffset + 3]
)
var alpha: UInt8 = 0
var red: UInt8 = 0
var green: UInt8 = 0
var blue: UInt8 = 0
switch componentLayout {
case .bgra:
alpha = components.3
red = components.2
green = components.1
blue = components.0
case .abgr:
alpha = components.0
red = components.3
green = components.2
blue = components.1
case .argb:
alpha = components.0
red = components.1
green = components.2
blue = components.3
case .rgba:
alpha = components.3
red = components.0
green = components.1
blue = components.2
default:
return .clear
}
// If chroma components are premultiplied by alpha and the alpha is `0`,
// keep the chroma components to their current values.
if cgImage.bitmapInfo.chromaIsPremultipliedByAlpha && alpha != 0 {
let invUnitAlpha = 255/CGFloat(alpha)
red = UInt8((CGFloat(red)*invUnitAlpha).rounded())
green = UInt8((CGFloat(green)*invUnitAlpha).rounded())
blue = UInt8((CGFloat(blue)*invUnitAlpha).rounded())
}
return .init(red: red, green: green, blue: blue, alpha: alpha)
} else if componentLayout.count == 3 {
let components = (
dataPtr[pixelOffset + 0],
dataPtr[pixelOffset + 1],
dataPtr[pixelOffset + 2]
)
var red: UInt8 = 0
var green: UInt8 = 0
var blue: UInt8 = 0
switch componentLayout {
case .bgr:
red = components.2
green = components.1
blue = components.0
case .rgb:
red = components.0
green = components.1
blue = components.2
default:
return .clear
}
return .init(red: red, green: green, blue: blue, alpha: UInt8(255))
} else {
assertionFailure("Unsupported number of pixel components")
return .clear
}
}
}
public extension UIColor {
convenience init(red: UInt8, green: UInt8, blue: UInt8, alpha: UInt8) {
self.init(
red: CGFloat(red)/255,
green: CGFloat(green)/255,
blue: CGFloat(blue)/255,
alpha: CGFloat(alpha)/255)
}
}
public extension CGBitmapInfo {
enum ComponentLayout {
case bgra
case abgr
case argb
case rgba
case bgr
case rgb
var count: Int {
switch self {
case .bgr, .rgb: return 3
default: return 4
}
}
}
var componentLayout: ComponentLayout? {
guard let alphaInfo = CGImageAlphaInfo(rawValue: rawValue & Self.alphaInfoMask.rawValue) else { return nil }
let isLittleEndian = contains(.byteOrder32Little)
if alphaInfo == .none {
return isLittleEndian ? .bgr : .rgb
}
let alphaIsFirst = alphaInfo == .premultipliedFirst || alphaInfo == .first || alphaInfo == .noneSkipFirst
if isLittleEndian {
return alphaIsFirst ? .bgra : .abgr
} else {
return alphaIsFirst ? .argb : .rgba
}
}
var chromaIsPremultipliedByAlpha: Bool {
let alphaInfo = CGImageAlphaInfo(rawValue: rawValue & Self.alphaInfoMask.rawValue)
return alphaInfo == .premultipliedFirst || alphaInfo == .premultipliedLast
}
}
UIImage is a wrapper the bytes are CGImage or CIImage
According the the Apple Reference on UIImage the object is immutable and you have no access to the backing bytes. While it is true that you can access the CGImage data if you populated the UIImage with a CGImage (explicitly or implicitly), it will return NULL if the UIImage is backed by a CIImage and vice-versa.
Image objects not provide direct access to their underlying image
data. However, you can retrieve the image data in other formats for
use in your app. Specifically, you can use the cgImage and ciImage
properties to retrieve versions of the image that are compatible with
Core Graphics and Core Image, respectively. You can also use the
UIImagePNGRepresentation(:) and UIImageJPEGRepresentation(:_:)
functions to generate an NSData object containing the image data in
either the PNG or JPEG format.
Common tricks to getting around this issue
As stated your options are
UIImagePNGRepresentation or JPEG
Determine if image has CGImage or CIImage backing data and get it there
Neither of these are particularly good tricks if you want output that isn't ARGB, PNG, or JPEG data and the data isn't already backed by CIImage.
My recommendation, try CIImage
While developing your project it might make more sense for you to avoid UIImage altogether and pick something else. UIImage, as a Obj-C image wrapper, is often backed by CGImage to the point where we take it for granted. CIImage tends to be a better wrapper format in that you can use a CIContext to get out the format you desire without needing to know how it was created. In your case, getting the bitmap would be a matter of calling
- render:toBitmap:rowBytes:bounds:format:colorSpace:
As an added bonus you can start doing nice manipulations to the image by chaining filters onto the image. This solves a lot of the issues where the image is upside down or needs to be rotated/scaled etc.
Building on Olie and Algal's answer, here is an updated answer for Swift 3
public func getRGBAs(fromImage image: UIImage, x: Int, y: Int, count: Int) -> [UIColor] {
var result = [UIColor]()
// First get the image into your data buffer
guard let cgImage = image.cgImage else {
print("CGContext creation failed")
return []
}
let width = cgImage.width
let height = cgImage.height
let colorSpace = CGColorSpaceCreateDeviceRGB()
let rawdata = calloc(height*width*4, MemoryLayout<CUnsignedChar>.size)
let bytesPerPixel = 4
let bytesPerRow = bytesPerPixel * width
let bitsPerComponent = 8
let bitmapInfo: UInt32 = CGImageAlphaInfo.premultipliedLast.rawValue | CGBitmapInfo.byteOrder32Big.rawValue
guard let context = CGContext(data: rawdata, width: width, height: height, bitsPerComponent: bitsPerComponent, bytesPerRow: bytesPerRow, space: colorSpace, bitmapInfo: bitmapInfo) else {
print("CGContext creation failed")
return result
}
context.draw(cgImage, in: CGRect(x: 0, y: 0, width: width, height: height))
// Now your rawData contains the image data in the RGBA8888 pixel format.
var byteIndex = bytesPerRow * y + bytesPerPixel * x
for _ in 0..<count {
let alpha = CGFloat(rawdata!.load(fromByteOffset: byteIndex + 3, as: UInt8.self)) / 255.0
let red = CGFloat(rawdata!.load(fromByteOffset: byteIndex, as: UInt8.self)) / alpha
let green = CGFloat(rawdata!.load(fromByteOffset: byteIndex + 1, as: UInt8.self)) / alpha
let blue = CGFloat(rawdata!.load(fromByteOffset: byteIndex + 2, as: UInt8.self)) / alpha
byteIndex += bytesPerPixel
let aColor = UIColor(red: red, green: green, blue: blue, alpha: alpha)
result.append(aColor)
}
free(rawdata)
return result
}
swift
To access the raw RGB values of an UIImage in Swift 5 use the underlying CGImage and its dataProvider:
import UIKit
let image = UIImage(named: "example.png")!
guard let cgImage = image.cgImage,
let data = cgImage.dataProvider?.data,
let bytes = CFDataGetBytePtr(data) else {
fatalError("Couldn't access image data")
}
assert(cgImage.colorSpace?.model == .rgb)
let bytesPerPixel = cgImage.bitsPerPixel / cgImage.bitsPerComponent
for y in 0 ..< cgImage.height {
for x in 0 ..< cgImage.width {
let offset = (y * cgImage.bytesPerRow) + (x * bytesPerPixel)
let components = (r: bytes[offset], g: bytes[offset + 1], b: bytes[offset + 2])
print("[x:\(x), y:\(y)] \(components)")
}
print("---")
}
https://www.ralfebert.de/ios/examples/image-processing/uiimage-raw-pixels/
Based on different answers but mainly on this, this works for what I need:
UIImage *image1 = ...; // The image from where you want a pixel data
int pixelX = ...; // The X coordinate of the pixel you want to retrieve
int pixelY = ...; // The Y coordinate of the pixel you want to retrieve
uint32_t pixel1; // Where the pixel data is to be stored
CGContextRef context1 = CGBitmapContextCreate(&pixel1, 1, 1, 8, 4, CGColorSpaceCreateDeviceRGB(), kCGImageAlphaNoneSkipFirst);
CGContextDrawImage(context1, CGRectMake(-pixelX, -pixelY, CGImageGetWidth(image1.CGImage), CGImageGetHeight(image1.CGImage)), image1.CGImage);
CGContextRelease(context1);
As a result of this lines, you will have a pixel in AARRGGBB format with alpha always set to FF in the 4 byte unsigned integer pixel1.

Resources