举个🌰
一个培养皿里有若干条鱼苗,需要将它全部区分识别出来,
像如下图所示的小蝌蚪就是(培养皿里三个黑点是热带鱼苗,做实验用的,一毫米长)
用的是海康威视的黑白工业相机拍摄。
先讲讲思路,图片是一组庞大的矩阵数据,每一个像素点有用数据为五个分别为RGB(三原色),以及XY坐标。也就是说我们能将整张图片每一个像素点的数据提取出来加以分析。那么就可以做到图片识别。
源代码会在最下方贴出。
首先整个流程思想是这杨
#region 6孔混合鱼苗VisionHelper.Two_Level(@"C:\Users\Administrator\Desktop\右上角三只鱼的提取\小鱼图片2-单个.png", @"C:\Users\Administrator\Desktop\3.1\二级化.png");VisionHelper.Outline(@"C:\Users\Administrator\Desktop\3.1\二级化.png", @"C:\Users\Administrator\Desktop\3.1\轮廓检测.png");VisionHelper.CutCircle(@"C:\Users\Administrator\Desktop\3.1\轮廓检测.png", @"C:\Users\Administrator\Desktop\3.1\圆形剪切.bmp");VisionHelper.ExtractCircle(@"C:\Users\Administrator\Desktop\3.1\圆形剪切.bmp", @"C:\Users\Administrator\Desktop\3.1\圆形提取.jpg");var data = VisionHelper.GetImagePixel(@"C:\Users\Administrator\Desktop\3.1\圆形提取.jpg");data = VisionHelper.FishExtract(data, @"C:\Users\Administrator\Desktop\3.1\圆形提取.jpg");var fish = VisionHelper.FishGroup(data);fish = VisionHelper.FishDistinct(fish);data = VisionHelper.FishCenter(fish);#endregion
先将抓取的图片二级化,效果如下所示
VisionHelper.Two_Level(@"C:\Users\Administrator\Desktop\右上角三只鱼的提取\小鱼图片2-单个.png", @"C:\Users\Administrator\Desktop\3.1\二级化.png");
原图左,处理图右
二级化以后,这张图片的数据就只剩黑色和白色,如果二级化时没有损坏到目标特征像素点,那么接下来提取目标特征像素点会很容易,因为只有黑白两色
接下来做轮廓监测,将整个培养皿扫描出来并且去除,这杨就只剩培养皿内的鱼苗和食物或者排泄物
VisionHelper.Outline(@"C:\Users\Administrator\Desktop\3.1\二级化.png", @"C:\Users\Administrator\Desktop\3.1\轮廓检测.png");
效果如下所示
在图片处理的算法中,我用红圈标注了培养皿内的区域,并且用蓝点打出了中心
接下来呢,可以将其他无用的图片区域全部剪切掉,就是图片内圆形切割
VisionHelper.CutCircle(@"C:\Users\Administrator\Desktop\3.1\轮廓检测.png", @"C:\Users\Administrator\Desktop\3.1\圆形剪切.bmp");
切割效果如下图所示
因为当初代码里设定生成的图片是BMP,上传不了博客,所以这粗糙的截图一下。
可以看到圆形剪切.bmp里只剩培养皿内区域的图片了
之前轮廓处理和圆形剪切形成的红色,蓝色圆圈或者中心点代码里可以设置不写入
那么接下来就是对圆形剪切区域的有用像素进行提取和分析
var data = VisionHelper.GetImagePixel(@"C:\Users\Administrator\Desktop\3.1\圆形提取.jpg");data = VisionHelper.FishExtract(data, @"C:\Users\Administrator\Desktop\3.1\圆形提取.jpg");var fish = VisionHelper.FishGroup(data);fish = VisionHelper.FishDistinct(fish);data = VisionHelper.FishCenter(fish);
我先展示下最终的结果
经过我进行数据处理后的图片内提取出了三条鱼的中心点位数据
我们校验一下答案
下图1是原图
三个小黑点是三条鱼数据正确,坐标是否正确?我用画图工具打开校验
如下三图所示,为了更直观的展示结果,用鼠标浮在指定坐标,手机拍摄的,不是很清楚但是看得清,大家可以双击图片放大
第一条数据151,77,在图内鼠标右上角指向的小鱼苗内
第二条数据22,88,在图内鼠标左侧指向的小鱼苗内
第三条数据137,148,88,在图内鼠标右下角指向的小鱼苗内
接下来贴出我手搓的核心算法
整个VisionHelper运用了OpenCvSharp和Emgu.CV这两个第三方图片处理框架的算法,所有的方法都可以灵活运用,方法体内的参数可以随着实际需要识别的物体做调整
(源代码里有那么多注释应该就不用在讲解基础框架和算法应用了吧,嘻嘻)
using Emgu.CV; using Emgu.CV.CvEnum; using Emgu.CV.Structure; using OpenCvSharp; using System.Drawing.Drawing2D; using System.Drawing.Imaging; using FishVision.Model;namespace FishVision {public class VisionHelper{/// <summary>/// 1.二级化 /// </summary>/// <param name="oldpath"></param>/// <param name="newPath"></param>public static void Two_Level(string oldpath,string newPath){Emgu.CV.Mat image = CvInvoke.Imread(oldpath, Emgu.CV.CvEnum.ImreadModes.Grayscale);Emgu.CV.Mat mid = new Emgu.CV.Mat();CvInvoke.Threshold(image, mid, 125, 255, ThresholdType.Binary);//180 改 CvInvoke.Imwrite(newPath, mid);}/// <summary>/// 2.轮廓检测/// </summary>public static void Outline(string oldpath, string newPath){//读取图片var img = Cv2.ImRead(oldpath);//转换成灰度图OpenCvSharp.Mat gray = img.CvtColor(ColorConversionCodes.BGR2GRAY);//阈值操作 阈值参数可以用一些可视化工具来调试得到OpenCvSharp.Mat ThresholdImg = gray.Threshold(135, 255, ThresholdTypes.Binary);//Cv2.ImShow("Threshold", ThresholdImg);//降噪 高斯变化//Mat gaussImg= ThresholdImg.GaussianBlur(new Size(5, 5), 0.8);//Cv2.ImShow("GaussianBlur", gaussImg);//中值滤波降噪//Mat medianImg = ThresholdImg.MedianBlur(5);//Cv2.ImShow("MedianBlur", medianImg);//膨胀+腐蚀//Mat kernel = new Mat(15, 15, MatType.CV_8UC1);//Mat DilateImg = ThresholdImg.Dilate(kernel);////腐蚀处理//Mat binary = DilateImg.Erode(kernel); OpenCvSharp.Mat element = Cv2.GetStructuringElement(MorphShapes.Ellipse, new OpenCvSharp.Size(3, 3));OpenCvSharp.Mat openImg = ThresholdImg.MorphologyEx(MorphTypes.Open, element);//Cv2.ImShow("Dilate & Erode", openImg);//设置感兴趣的区域int x = 0, y = 0, w = img.Width, h = img.Height;Rect roi = new Rect(x, y, w, h);OpenCvSharp.Mat ROIimg = new OpenCvSharp.Mat(openImg, roi);//Cv2.ImShow("ROI Image", ROIimg);//寻找图像轮廓 OpenCvSharp.Point[][] contours;HierarchyIndex[] hierachy;Cv2.FindContours(ROIimg, out contours, out hierachy, RetrievalModes.List, ContourApproximationModes.ApproxTC89KCOS);//根据找到的轮廓点,拟合椭圆for (int i = 0; i < contours.Length; i++){//拟合函数必须至少5个点,少于则不拟合if (contours[i].Length < 150 || contours[i].Length > 200) continue;//椭圆拟合var rrt = Cv2.FitEllipse(contours[i]);//ROI复原rrt.Center.X += x;rrt.Center.Y += y;//画椭圆Cv2.Ellipse(img, rrt, new Scalar(0, 0, 255), 2, LineTypes.AntiAlias);//画圆心Cv2.Circle(img, (int)(rrt.Center.X), (int)(rrt.Center.Y), 4, new Scalar(255, 0, 0), -1, LineTypes.Link8, 0);}//Cv2.ImShow("Fit Circle", img); Cv2.ImWrite(newPath, img);}/// <summary>/// 3.圆形剪切/// </summary>/// <param name="oldpath"></param>/// <param name="newPath"></param>public static void CutCircle(string oldpath, string newPath){Image<Bgr, Byte> src = new Image<Bgr, byte>(oldpath);int scale = 1;if (src.Width > 500){scale = 2;}if (src.Width > 1000){scale = 10;}if (src.Width > 10000){scale = 100;}var size = new System.Drawing.Size(src.Width / scale, src.Height / scale);Image<Bgr, Byte> srcNewSize = new Image<Bgr, byte>(size);CvInvoke.Resize(src, srcNewSize, size);//将图像转换为灰度Emgu.CV.UMat grayImage = new Emgu.CV.UMat();CvInvoke.CvtColor(srcNewSize, grayImage, ColorConversion.Bgr2Gray);//使用高斯滤波去除噪声CvInvoke.GaussianBlur(grayImage, grayImage, new System.Drawing.Size(3, 3), 3);//霍夫圆检测CircleF[] circles = CvInvoke.HoughCircles(grayImage, Emgu.CV.CvEnum.HoughModes.Gradient, 2.0, 200.0, 100.0, 180.0, 5);Rectangle rectangle = new Rectangle();float maxRadius = 0;foreach (CircleF circle in circles){var center = circle.Center;//圆心var radius = circle.Radius;//半径if (radius > maxRadius){maxRadius = radius;rectangle = new Rectangle((int)(center.X - radius) * scale,(int)(center.Y - radius) * scale,(int)radius * 2 * scale + scale,(int)radius * 2 * scale + scale);}srcNewSize.Draw(circle, new Bgr(System.Drawing.Color.Blue), 4);}//CvInvoke.Imwrite("原始图片.bmp", srcNewSize); //保存原始图片if (maxRadius == 0){//MessageBox.Show("没有圆形"); }CvInvoke.cvSetImageROI(srcNewSize.Ptr, rectangle);//设置兴趣点—ROI(region of interest )var clone = srcNewSize.Clone();CvInvoke.Imwrite(newPath, clone); //保存结果图 src.Dispose();srcNewSize.Dispose();grayImage.Dispose();}/// <summary>/// 4.圆形提取/// </summary>public static void ExtractCircle(string oldpath, string newPath){// 加载原始图片Bitmap originalImage = new Bitmap(oldpath);int diameter = Math.Min(originalImage.Width, originalImage.Height); // 获取最小边长作为直径int x = (originalImage.Width - diameter) / 2; // 计算起始x坐标int y = (originalImage.Height - diameter) / 2; // 计算起始y坐标// 创建与圆形大小相等的bitmapBitmap croppedImage = new Bitmap(diameter, diameter);using (Graphics g = Graphics.FromImage(croppedImage)){g.Clear(Color.LightBlue); // 设置圆圈外的颜色// 设置高质量插值法g.InterpolationMode = InterpolationMode.HighQualityBicubic;// 设置高质量,低速度呈现平滑程度g.SmoothingMode = SmoothingMode.HighQuality;g.PixelOffsetMode = PixelOffsetMode.HighQuality;g.CompositingQuality = CompositingQuality.HighQuality;// 创建一个圆形路径using (GraphicsPath path = new GraphicsPath()){path.AddEllipse(0, 0, diameter, diameter);// 设置裁剪区域为圆形路径 g.SetClip(path);// 从原始图片中绘制圆形区域到新图片g.DrawImage(originalImage, new Rectangle(0, 0, diameter, diameter), x, y, diameter, diameter, GraphicsUnit.Pixel);}}// 保存剪切后的图片 croppedImage.Save(newPath, ImageFormat.Jpeg);}/// <summary>/// 5.像素提取(默认黑像素)/// </summary>/// <param name="img"></param>/// <returns></returns>public static List<string> GetImagePixel(string oldpath)//过滤 {// 加载原始图片Bitmap img = new Bitmap(oldpath);//0 黑色//95 深灰//240 浅灰//255 白 List<int> R = new List<int>();List<int> G = new List<int>();List<int> B = new List<int>();List<string> xyList = new List<string>();for (int y = 0; y < img.Height; y++){for (int x = 0; x < img.Width; x++){var a = img.GetPixel(x, y);if (a.R == 0 && a.G == 0 && a.B == 0){R.Add(img.GetPixel(x, y).R);G.Add(img.GetPixel(x, y).G);B.Add(img.GetPixel(x, y).B);xyList.Add(x + "|" + y);}}}return xyList;}/// <summary>/// 6.鱼像素提取/// </summary>/// <returns></returns>public static List<string> FishExtract(List<string> data ,string circleImg){for (int i = 0; i < data.Count; i++){var str = data[i].Split("|");if (!string.IsNullOrWhiteSpace(data[i])){Bitmap image2 = new Bitmap(circleImg);//周边检测 6var list = GetSurroundingPixels(image2, Convert.ToInt32(str[0]), Convert.ToInt32(str[1]));if (list.Where(a => a.R == 0 && a.G == 0 && a.B == 0).Count() >= 2)//这是鱼像素特征 {}else{data[i] = string.Empty;//非鱼 }}}return data;}//像素周边检测public static List<FishVision.Model.Pixel> GetSurroundingPixels(Bitmap bitmap, int x, int y){var result = new List<FishVision.Model.Pixel>();int width = bitmap.Width;int height = bitmap.Height;Color[,] surroundingPixels = new Color[3, 3]; // 3x3 grid including the center pixelfor (int i = -1; i <= 1; i++) // Loop through the 3x3 grid around the center pixel {for (int j = -1; j <= 1; j++){int newX = x + i;int newY = y + j;// Check if the new coordinates are within the bounds of the imageif (newX >= 0 && newX < width && newY >= 0 && newY < height){surroundingPixels[i + 1, j + 1] = bitmap.GetPixel(newX, newY);}else{// Optionally, set out-of-bounds pixels to a default color or handle them as neededsurroundingPixels[i + 1, j + 1] = Color.Transparent; // or any other color you prefer }}}// Use surroundingPixels as needed (e.g., print colors)for (int i = 0; i < 3; i++) // Printing the surrounding pixels for demonstration purposes {for (int j = 0; j < 3; j++){var model = new FishVision.Model.Pixel();model.X = x + i - 1;model.Y = y + j - 1;model.R = surroundingPixels[i, j].R;model.G = surroundingPixels[i, j].G;model.B = surroundingPixels[i, j].B;result.Add(model);//Console.WriteLine($"Pixel ({x + i - 1}, {y + j - 1}): {surroundingPixels[i, j]}"); }}return result;}/// <summary>/// 7.鱼像素去重/// </summary>/// <param name="listFish"></param>/// <returns></returns>public static List<FishModel> FishDistinct(List<FishModel> listFish){if (listFish != null && listFish.Count() > 0){for (int i = 0; i < listFish.Count; i++){listFish[i].FishIndex = listFish[i].FishIndex.Distinct().ToList();}}return listFish;}/// <summary>/// 8.鱼像素分组/// </summary>public static List<FishModel> FishGroup(List<string> data){var fish = new List<FishModel>();//鱼像素分组for (int i = 0; i < data.Count; i++){for (int b = 0; b < data.Count; b++){if (!string.IsNullOrWhiteSpace(data[i]) && !string.IsNullOrWhiteSpace(data[b])){var data_i_xy = data[i].Split("|");var data_b_xy = data[b].Split("|");if (AreAdjacent(Convert.ToInt32(data_i_xy[0]), Convert.ToInt32(data_i_xy[1]), Convert.ToInt32(data_b_xy[0]), Convert.ToInt32(data_b_xy[1])))//相邻的鱼像素合并一组 {var entity = fish.Where(a => a.FishIndex.Contains(data[i]) || a.FishIndex.Contains(data[b])).FirstOrDefault();if (entity != null){entity.FishIndex.Add(data[i]);entity.FishIndex.Add(data[b]);}else{FishModel model = new FishModel();model.FishIndex = new List<string>();model.FishIndex.Add(data[i]);model.FishIndex.Add(data[b]);fish.Add(model);}}}}}return fish;}/// <summary>/// 像素是否相邻/// </summary>/// <param name="x1"></param>/// <param name="y1"></param>/// <param name="x2"></param>/// <param name="y2"></param>/// <returns></returns>public static bool AreAdjacent(int x1, int y1, int x2, int y2){// 检查x和y坐标之差是否为1,这样可以确保像素是直接相邻的return (Math.Abs(x1 - x2) <= 1 && Math.Abs(y1 - y2) <= 1) && !(x1 == x2 && y1 == y2);}/// <summary>/// 9.每条鱼的像素群寻找中位值作为轨迹坐标/// </summary>/// <param name="listFish"></param>/// <returns></returns>public static List<string> FishCenter(List<FishModel> listFish){var result = new List<string>();if (listFish != null && listFish.Count() > 0){for (int i = 0; i < listFish.Count; i++){//取中位值var index = -1;if (listFish[i].FishIndex.Count() % 2 == 0) // 偶数长度 {index = listFish[i].FishIndex.Count() / 2;}else // 奇数长度 {index = (listFish[i].FishIndex.Count() + 1) / 2;}//这条鱼中心坐标var center = listFish[i].FishIndex[index];result.Add(center);}}return result;}/// <summary>/// 反向二级化/// </summary>/// <param name="oldpath"></param>/// <param name="newPath"></param>public static void Two_LevelReversal(string oldpath, string newPath){OpenCvSharp.Mat src = Cv2.ImRead(oldpath, OpenCvSharp.ImreadModes.Grayscale);OpenCvSharp.Mat dst = new OpenCvSharp.Mat();// 反向二值化:大于 127 的像素设为 0,其他设为 255Cv2.Threshold(src, dst, 135, 255, ThresholdTypes.BinaryInv);Cv2.ImWrite(newPath, dst);}/// <summary>/// 96孔鱼苗高光二级化处理/// </summary>/// <param name="oldpath"></param>/// <param name="newPath"></param>public static void Two_LevelHeight(string oldpath, string newPath){Emgu.CV.Mat image = CvInvoke.Imread(oldpath, Emgu.CV.CvEnum.ImreadModes.Grayscale);Emgu.CV.Mat mid2 = new Emgu.CV.Mat();CvInvoke.Threshold(image, mid2, 180, 255, ThresholdType.Binary);CvInvoke.Imwrite(newPath, mid2);}/// <summary>/// 斑点检测(用不着)/// </summary>/// <param name="mat">图片</param>/// <param name="resultMat">结果图片</param>/// <returns>斑点中心点数据</returns>public static KeyPoint[] SimpleblobDetector(OpenCvSharp.Mat mat, out OpenCvSharp.Mat resultMat){// 转化为灰度图OpenCvSharp.Mat gray = new OpenCvSharp.Mat();Cv2.CvtColor(mat, gray, ColorConversionCodes.BGR2GRAY);// 创建SimpleBlobDetector并设置参数OpenCvSharp.SimpleBlobDetector.Params parameters = new OpenCvSharp.SimpleBlobDetector.Params();parameters.BlobColor = 0;//斑点的亮度值,取值为0或255,默认为0,表示只检测黑色斑点。parameters.FilterByArea = true; // 是否根据斑点的面积进行过滤,默认为trueparameters.MinArea = 10; // 最小的斑点面积,默认为25parameters.MaxArea = 6000; // 最大的斑点面积,默认为5000// 创建SimpleBlobDetectorOpenCvSharp.SimpleBlobDetector detector = OpenCvSharp.SimpleBlobDetector.Create(parameters);// 检测斑点KeyPoint[] keypoints = detector.Detect(gray);// 在图像上绘制斑点resultMat = new OpenCvSharp.Mat();Cv2.DrawKeypoints(mat, keypoints, resultMat, Scalar.All(-1));return keypoints;}} }