|
【U020】如何实现视觉识别
作者:机器谱
VS配置OpenCV 识别颜色 识别形状 |
(2)在“解决方案资源管理器”里右击“引用”,“添加引用”,点击左侧的“浏览”,再点击下图中左侧的“浏览”选项,添加dll文件。
一、实验内容
在Visual Studio 2015.net下配置OpenCV环境,便于后续视觉相关实验的开发。
二、实验设备
计算机一台。
三、操作步骤
1、配置环境变量
(1)打开配置环境资料包\OpenCV3.2.zip,解压到任意盘符目录,如 E:\Workspace\OpenCV_lib_3.2
(2)配置环境变量:在“系统属性”下“高级”选项找到“环境变量”,点击进入,在“系统变量”Path变量添加OpenCV以及OpenCvSharp变量,如下图所示:
① E:\Workspace\OpenCV_lib_3.2\OpenCvSharp-3.2.0-x64-20171112
② E:\Workspace\OpenCV_lib_3.2\OpenCvSharp-3.2.0-x64-20171112\DebuggerVisualizers
③ E:\Workspace\OpenCV_lib_3.2\opencv\build\x64\vc14\bin
点击“浏览”选项,添加所需的dll文件,路径在:E:\Workspace\OpenCV_lib_3.2\OpenCvSharp-3.2.0-x64-20171112\net40
(3)重启电脑,使环境变量配置生效。
2、C#工程配置
(1)新建C#控制台程序
(3)在“配置管理器”中,将下面“平台”选择“x64"。
using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using OpenCvSharp; namespace opencv_test { class Program { static void Main(string[] args) { Console.WriteLine("*************************************************按q退出程序*************************************************"); while (true) { Mat SRC = Cv2.ImRead("lena.jpg"); if (SRC.Empty()) { Console.WriteLine("SRC is empty"); } Cv2.NamedWindow("SRC", 0); Cv2.ImShow("SRC", SRC); Char key = (Char)Cv2.WaitKey(10); if (key == 'q') break; } } } } |
至此,OpenCV的环境配置完成,可进行后续视觉相关实验的开发。
视觉相关实验案例可参考 【R023】小型双轮差速底盘-视觉循迹 、【R325】小黑仿生轮腿机器人-机器视觉 。
注:
若运行过程中出现错误,原因有下:
① VS“配置管理器”里的平台选择错误
② “Debug”目录下未放置图片,导致抛出异常
四、资料清单
序号 | 内容 |
1 | 配置环境资料包 |
(5)程序执行,运行如下:
(4)测试程序:功能为加载一张图片并通过OpenCV的窗口显示出来。
在Debug目录(如:E:\Projectcode\C#\opencv_test\opencv_test\bin\x64\Debug)下放置一张图片,此处命名为 ”lena.jpg”
参考程序examples\1\opencv_test\opencv_test\Program.cs如下:
【整体打包】-【U020】如何实现视觉识别-VS配置OpenCV环境-资料附件.zip | 293.04MB | 下载30次 | 下载 |
1. 功能说明
通过摄像头识别特定颜色(红、绿、蓝)。摄像头采集图像信息并通过WiFi将信息传递给PC端,然后PC端根据比例判断出目标颜色在色盘上的所属颜色后,指针便会指向对应颜色。
红、绿、蓝-色块
2. 电子硬件
本实验中采用了以下硬件:
using System; using System.IO; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using System.Windows; using System.Windows.Controls; using System.Windows.Data; using System.Windows.Documents; using System.Windows.Input; using System.Windows.Media; using System.Windows.Media.Imaging; using System.Windows.Navigation; using System.Windows.Shapes; using System.Windows.Media.Animation; using System.Threading; using OpenCvSharp; using System.Drawing; using System.Drawing.Imaging; namespace Color_Detect { /// <summary> /// Color_Detect /// </summary> public partial class MainWindow : System.Windows.Window { /* * 指针角度对应各颜色 * 25 -> 红色 * 90 -> 绿色 * 150 -> 蓝色 */ int ANGLE_RED = 0; int ANGLE_GREEN = 0; int ANGLE_BLUE = 0; //各颜色像素所占窗口的比例 double numOfred = 0.0; double numOfgreen = 0.0; double numOfblue = 0.0; //创建视频图像实例 VideoCapture capture = new VideoCapture("http://192.168.8.1:8083/?action=stream"); Mat frame = new Mat(); //存储视频每一帧图像像素 Mat resultColor = new Mat(); //存储检测后的颜色像素 //视频显示切换变量 Boolean isChange = false; public MainWindow() { InitializeComponent(); } private void Window_Loaded(object sender, RoutedEventArgs e) { ANGLE_RED = 25; ANGLE_GREEN = 90; ANGLE_BLUE = 150; } //颜色指示动画函数 int angelCurrent = 0; private void ColorIndicate(int where) { RotateTransform rt = new RotateTransform(); rt.CenterX = 150; rt.CenterY = 185; this.indicatorPin.RenderTransform = rt; double timeAnimation = Math.Abs(angelCurrent - where) * 5; DoubleAnimation da = new DoubleAnimation(angelCurrent, where, new Duration(TimeSpan.FromMilliseconds(timeAnimation))); da.AccelerationRatio = 0.8; rt.BeginAnimation(RotateTransform.AngleProperty, da); switch (where) { case 25: colorDisplay.Content = "红色"; break; case 90: colorDisplay.Content = "绿色"; break; case 150: colorDisplay.Content = "蓝色"; break; default: colorDisplay.Content = "颜色指示"; break; } angelCurrent = where; } /// <summary> /// MatToBitmap(Mat image) /// </summary> public static Bitmap MatToBitmap(Mat image) { return OpenCvSharp.Extensions.BitmapConverter.ToBitmap(image); } /// <summary> /// BitmapToBitmapImage(System.Drawing.Bitmap bitmap) /// </summary> public static BitmapImage BitmapToBitmapImage(Bitmap bitmap) { using (MemoryStream stream = new MemoryStream()) { bitmap.Save(stream, ImageFormat.Png); //格式选Bmp时,不带透明度 stream.Position = 0; BitmapImage result = new BitmapImage(); result.BeginInit(); // According to MSDN, "The default OnDemand cache option retains access to the stream until the image is needed." // Force the bitmap to load right now so we can dispose the stream. result.CacheOption = BitmapCacheOption.OnLoad; result.StreamSource = stream; result.EndInit(); result.Freeze(); return result; } } //颜色检测函数 private void filterColor() { Mat hsvImage = frame.CvtColor(ColorConversionCodes.BGR2HSV); resultColor = new Mat(hsvImage.Rows, hsvImage.Cols, MatType.CV_8UC3, Scalar.All(255)); double H = 0.0, S = 0.0, V = 0.0; float area = (float)(hsvImage.Rows * hsvImage.Cols); float rateOfred = 0, rateOfgreen = 0, rateOfblue = 0; for (int i = 0; i < hsvImage.Rows; i++) { for (int j = 0; j < hsvImage.Cols; j++) { H = hsvImage.Get<Vec3b>(i, j)[0]; S = hsvImage.Get<Vec3b>(i, j)[1]; V = hsvImage.Get<Vec3b>(i, j)[2]; var color = frame.Get<Vec3b>(i, j); if (((H >= 0 && H <= 10) || (H >= 125 && H <= 180)) && S >= 43 && V >= 46) //红色像素所在hsv范围 { resultColor.Set<Vec3b>(i, j, color); numOfred++; } else if ((H >= 33 && H <= 83) && S >= 43 && V >= 46) //绿色像素所在hsv范围 { resultColor.Set<Vec3b>(i, j, color); numOfgreen++; } else if ((H > 100 && H < 124) && S >= 43 && V >= 46) //蓝色像素所在hsv范围 { resultColor.Set<Vec3b>(i, j, color); numOfblue++; } } } rateOfred = (float)(numOfred) / area * 100; rateOfgreen = (float)(numOfgreen) / area * 100; rateOfblue = (float)(numOfblue) / area * 100; if (rateOfred > 85) { ColorIndicate(ANGLE_RED); } else if (rateOfgreen > 85) { ColorIndicate(ANGLE_GREEN); } else if (rateOfblue > 85) { ColorIndicate(ANGLE_BLUE); } numOfred = 0; numOfgreen = 0; numOfblue = 0; } //视频显示函数 private void ThreadCapShow() { while (true) { try { capture.Read(frame); // same as cvQueryFrame if (frame.Empty()) break; this.Dispatcher.Invoke( new Action( delegate { if (isChange) { filterColor(); originImage.Source = BitmapToBitmapImage(MatToBitmap(resultColor)); resultColor = null; } else { originImage.Source = BitmapToBitmapImage(MatToBitmap(frame)); } } )); //Cv2.WaitKey(100); //bitimg = null; } catch { } } } //加载视频 private void loadBtn_Click(object sender, RoutedEventArgs e) { if (originImage.Source != null) return; Thread m_thread = new Thread(ThreadCapShow); m_thread.IsBackground = true; m_thread.Start(); } //切换视频显示,显示检测结果 private void changeBtn_Click(object sender, RoutedEventArgs e) { if (!isChange) { isChange = true; changeBtn.Content = "返回"; } else { isChange = false; changeBtn.Content = "切换"; //指针角度归零 ColorIndicate(0); } } } } |
3. 功能实现
工作原理:
① 摄像头采集图像信息;
② 通过WiFi将信息传递给PC端(VS2015配置的OpenCV环境);
③ 在PC端修改红色色域范围,用于判断摄像范围内的红色像素;
采用HSV颜色模型
④ 计算检测在显示的摄像范围内的红色像素区域所占比例=红色像素范围/显示的摄像范围;
⑤ 根据比例判断目标颜色在色盘上所属颜色;
⑥ 指针指向对应颜色。
3.1硬件连接
将摄像头与路由器连接,启动路由器,将PC连接到路由器的WIFI网络。
本实验不需要用到主控板作为下位机,可直接通过WiFi将图像信号传递给PC端,所以无需下位机编程。
主控板与WiFi正常连线,给WiFi路由器模块通电。
接线说明:
① 将2510通信转接板连接到扩展板的扩展坞上面;
② 找到1根USB线,一端连接到2510通信转接板接口上,另一端连接到WiFi路由器USB接口上;
③ 将摄像头线连接到WiFi路由器接口上。
3.2示例程序
下面提供一个可以进行3个颜色(红、绿、蓝)识别的参考例程(MainWindow.xaml.cs):
程序设定的颜色为红色、绿色、蓝色,可以使用色卡或者特定颜色的物体来检测。
注意:程序中的比例值设置为85%时,可以进行三种颜色的识别判断,建议测试的色块距离小一些,识别效果会更好。
4. 资料清单
序号 | 内容 |
1 | 【U020】-识别颜色-例程源代码 |
【整体打包】-【U020】如何实现视觉识别-识别颜色-资料附件.zip | 402.65KB | 下载12次 | 下载 |
1. 功能说明
通过摄像头识别圆形及矩形两种形状。
2. 电子硬件
本实验中采用了以下硬件:
vector<Vec3f> circles; HoughCircles(image, circles, HOUGH_GRADIENT, 2.0, image.rows/8, // change this value to detect circles with different distances to each other 200, 85, 0, 0 // change the last two parameters // (min_radius & max_radius) to detect larger circles ); |
//多边形检测,通过约束条件寻找矩形 static void findSquares(const Mat& image, vector<vector<Point> >& squares) { squares.clear(); Mat pyr, timg, gray0(image.size(), CV_8U), gray; // down-scale and upscale the image to filter out the noise pyrDown(image, pyr, Size(image.cols / 2, image.rows / 2)); pyrUp(pyr, timg, image.size()); vector<vector<Point> > contours; // find squares in every color plane of the image for (int c = 0; c < 3; c++) { int ch[] = { c, 0 }; mixChannels(&timg, 1, &gray0, 1, ch, 1); // try several threshold levels for (int l = 0; l < N; l++) { // hack: use Canny instead of zero threshold level. // Canny helps to catch squares with gradient shading if (l == 0) { // apply Canny. Take the upper threshold from slider // and set the lower to 0 (which forces edges merging) Canny(gray0, gray, 0, thresh, 5); // dilate canny output to remove potential // holes between edge segments dilate(gray, gray, Mat(), Point(-1, -1)); } else { // apply threshold if l!=0: // tgray(x,y) = gray(x,y) < (l+1)*255/N ? 255 : 0 gray = gray0 >= (l + 1) * 255 / N; } // find contours and store them all as a list findContours(gray, contours, RETR_LIST, CHAIN_APPROX_SIMPLE); vector<Point> approx; // test each contour for (size_t i = 0; i < contours.size(); i++) { // approximate contour with accuracy proportional // to the contour perimeter approxPolyDP(Mat(contours[i]), approx, arcLength(Mat(contours[i]), true)*0.02, true); // square contours should have 4 vertices after approximation // relatively large area (to filter out noisy contours) // and be convex. // Note: absolute value of an area is used because // area may be positive or negative - in accordance with the // contour orientation if (approx.size() == 4 && fabs(contourArea(Mat(approx))) > 1000 && isContourConvex(Mat(approx))) { double maxCosine = 0; for (int j = 2; j < 5; j++) { // find the maximum cosine of the angle between joint edges double cosine = fabs(angle(approx[j % 4], approx[j - 2], approx[j - 1])); maxCosine = MAX(maxCosine, cosine); } // if cosines of all angles are small // (all angles are ~90 degree) then write quandrange // vertices to resultant sequence if (maxCosine < 0.3) squares.push_back(approx); } } } } } |
using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading.Tasks; using System.Windows; using System.Windows.Controls; using System.Windows.Data; using System.Windows.Documents; using System.Windows.Input; using System.Windows.Media; using System.Windows.Media.Imaging; using System.Windows.Navigation; using System.Windows.Shapes; using System.Windows.Forms; using System.Runtime.InteropServices; using System.Threading; namespace ShapeDetect { /// <summary> /// 形状识别 /// </summary> public partial class MainWindow : Window { //定义检测模式 int IMAGE_MODE = 1, VIDEO_MODE = 2; private AutoResetEvent exitEvent; private Thread m_thread; //导入动态链接库 [DllImport("HoughCircles_DLL.dll")] //检测圆 public static extern System.UIntPtr HoughCircles([MarshalAs(UnmanagedType.LPStr)]string address, int detect_mode); [DllImport("SquareDetect_DLL.dll")] //检测矩形 public static extern void SquareDetector([MarshalAs(UnmanagedType.LPStr)]string address, int detect_mode); public MainWindow() { InitializeComponent(); } private void Window_Loaded(object sender, RoutedEventArgs e) { GetIni(); imgCheckBtn.IsChecked = true; circleCheckBtn.IsChecked = true; } //获取ini配置文件信息 private void GetIni() { ini_RW.FileName = System.Windows.Forms.Application.StartupPath + "\\Config.ini"; this.videoAddress.Text = ini_RW.ReadIni("VideoUrl", "videourl", ""); this.ipAddress.Text = ini_RW.ReadIni("ControlUrl", "controlUrl", ""); this.portBox.Text = ini_RW.ReadIni("ControlPort", "controlPort", ""); } //修改配置 private void setBtn_Click(object sender, RoutedEventArgs e) { ini_RW.WriteIni("VideoUrl", "videourl", this.videoAddress.Text); ini_RW.WriteIni("ControlUrl", "controlUrl", this.ipAddress.Text); ini_RW.WriteIni("ControlPort", "controlPort", this.portBox.Text); System.Windows.MessageBox.Show("配置成功!请重启程序以使配置生效。", "配置信息", MessageBoxButton.OK, MessageBoxImage.Information); //this.Close(); } //计数清零 private void BoxClean() { circleTextBox.Text = "0"; recTextBox.Text = "0"; } //打开图片地址 private void imgBtn_Click(object sender, RoutedEventArgs e) { try { BoxClean(); //WPF中,OpenFileDialog位于Microsoft.Win32名称空间 Microsoft.Win32.OpenFileDialog dialog = new Microsoft.Win32.OpenFileDialog(); dialog.Filter = "All files (*.*)|*.*|jpg files (*.jpg)|*.jpg|png files(*.png)|*.png"; if (dialog.ShowDialog() == true) { string path = dialog.FileName; imgAddressBox.Text = path; } } catch { }; } //检测形状判断 private void ShapeDetect(string address, int mode) { if (circleCheckBtn.IsChecked == true) { //System.Windows.MessageBox.Show("检测圆形"); circleTextBox.Text = HoughCircles(address, mode).ToString(); } else if (recCheckBtn.IsChecked == true) { //System.Windows.MessageBox.Show("检测矩形"); SquareDetector(address, mode); } } //图片检测 private void imgDetect() { if (imgAddressBox.Text == string.Empty) { System.Windows.MessageBox.Show( "图片地址为空,请选择一张图片", "警告", MessageBoxButton.OK, MessageBoxImage.Information ); return; } else { ShapeDetect(imgAddressBox.Text, IMAGE_MODE); } } //视频检测 private void videoDetect() { try { while (true) { this.Dispatcher.Invoke( new Action( delegate { string ip = this.videoAddress.Text; ShapeDetect(ip, VIDEO_MODE); } )); } } catch { }; } //判断检测为图片还是视频,开启形状检测 private void detectBtn_Click(object sender, RoutedEventArgs e) { BoxClean(); if (imgCheckBtn.IsChecked == true) { imgDetect(); } else if (videoCheckBtn.IsChecked == true) { try { m_thread = new Thread(new ThreadStart(videoDetect)); m_thread.Start(); } catch { }; } } //按esc键退出视频检测,结束线程 private void Window_KeyDown(object sender, System.Windows.Input.KeyEventArgs e) { if (e.Key == Key.Escape) { exitEvent.Set(); m_thread.Join(); } } } } |
3. 功能实现
工作原理:
① 导入一张图片或者通过WiFi传递摄像信息给PC;
② 在PC端使用OpenCV对图像转化为灰度图像;
③ 检测圆形和矩形。
检测圆形使用霍夫变换:
检测矩形:
3.1硬件连接
将摄像头与路由器连接,启动路由器,将PC连接到路由器的WIFI网络。
接线说明:
① 将2510通信转接板连接到扩展板的扩展坞上面;
② 用3根母对母杜邦线将2510通信转接板与WiFi路由器连接起来,GND-GND、RX-RX、TX-TX;
③ 找到1根USB线,一端连接到2510通信转接板接口上,另一端连接到WiFi路由器USB接口上;
④ 将摄像头线连接到WiFi路由器接口上。
3.2示例程序
下面提供一个可以进行识别圆形和矩形的参考例程(ShapeDetect\ShapeDetect\MainWindow.xaml.cs):
程序识别圆形及矩形两种形状,包括对图像以及视频中的物体的形状检测,可参考上面的演示视频进行操作。图片中物体的形状识别,文末资料下载中提供一张测试图片,然后选择图片按钮,选择要检测的圆形或者矩形,点击形状检测。
视频中的形状识别,选择视频按钮,选择要检测的圆形或者矩形,点击形状检测,可以使用球体或者矩形状物体进行检测。
4. 资料清单
序号 | 内容 |
1 | 【U020】-识别形状-例程源代码 |
2 | 测试图片.jpg |
【整体打包】-【U020】如何实现视觉识别-识别形状-资料附件.zip | 63.47MB | 下载6次 | 下载 |
识别形状