With the development of artificial intelligence, big data, machine learning, computer vision and other technologies, convolutional neural network (CNN) is increasingly used in the field of image recognition, which greatly improves the efficiency and accuracy of recognition. Machine learning is data-driven and requires large amounts of data as a basis for experimentation. The richness and diversity of image data sets are crucial to the performance and expressive ability of convolutional neural network models. However, the existing fish image data set resources are relatively scarce, and the training set and test set samples are severely lacking. This makes it difficult to train neural network models, and it is difficult to meet the needs of deep convolutional neural network model optimization and performance improvement. A basic image data set for fish species classification was constructed by using a combination of web crawlers and manual camera collection in the laboratory. Larimichthys crocea, Hypophthalmichthys molitrix, Cyprinus carpio, Cololabis saira and Aristichthys nobilis were used as the test objects in this paper. First, we used web crawlers on the web to obtain pictures of these species of fish, and then, in a laboratory environment, we used cameras to take a large number of photos of these species of fish. In view of the problems of different scales and uncertain formats of images, image batch processing, unified data preprocessing was performed on all the acquired images, and the basic data set was enhanced through content transformation and scale transformation. The dataset was further enriched through this process and the image collection and induction of 7 993 samples were completed. On the basis of parameter sharing and local connectivity, a convolutional neural network model for fish recognition is constructed; the ReLU function was used as the activation function to improve the performance of the algorithm; the dropout and regularization were used to avoid overfitting. The test results showed that: the convolutional neural network fish species recognition model constructed in this study have good recognition accuracy and generalization ability. As the number of iterations increased, the performance of the convolutional neural network model gradually improved. It reached the best when the number of iterations came to 1 000. The accuracy of the model was 96.56%. The model adopted the machine learning method of supervised learning. Based on the CNN model, it realized the classification of five common fish species, with high recognition accuracy and good stability. The model has provided a new theoretical calculation model for the species identification of farmed fish.