A Cost-Effective Pigsty Monitoring System Based on a Video Sensor

Abstract

Automated activity monitoring has become important in many applications. In particular, automated monitoring is an important issue in large-scale management of group-housed livestock because it can save a significant part of farm workers' time or minimize the damage caused by livestock problems. In this paper, we propose an automated solution for measuring the daily-life activities of pigs by using video data in order to manage the group-housed pigs. Especially, we focus on the circadian rhythm of group-housed pigs under windowless and 24-hour light-on conditions. Also, we derive a cost-effective solution within the acceptable range of quality for the activity monitoring application. From the experimental results with the video monitoring data obtained from two pig farms, we believe our method based on circadian rhythm can be applied for detecting management problems of group-housed pigs in a cost-effective way.

1. Introduction

Early detection of management problems is an important issue in caring for group-housed livestock [1]. A real-time analysis of pig activity could provide useful information for the early detection of management problems. In particular, the damage caused by the recent outbreak of livestock diseases in Korea such as foot-and-mouth disease was serious (about 8 million pigs were buried) [2]. In contrast, caring for individual pigs by a few farm workers in a large-scale pig farm is almost impossible. For example, a pig farm where we obtained video monitoring data has more than 20,000 pigs and 10 farm workers, whereas another farm has more than 5,000 pigs and 2 farm workers. Caring for these pigs with few farm workers is infeasible, and an automated and cost-effective analysis of the daily-life activity is required for a large-scale pig farm.

For managing group-housed livestock, the convergence of IT and agriculture resulted in a new field of study of computers and electronics in agriculture. The convergence technology has been actively performed and found in various forms in the pigsty monitoring field. Kruse et al. [3] clarified whether wavelet analysis could identify water intake variation due to health problems and to differentiate between healthy and treated sows using the example of lactating sows. Guarino et al. [4] proposed an intelligent alarm system for the early detection of diseases using the continuous on-line monitoring of cough sounds. Ostersen et al. [5] suggested an alarm system for detecting oestrus in sows in the gestation section. Each sow carried an RFID-tag, and the detection is based on monitoring of the sows’ visits to a boar, where the duration and frequency of visits are modeled separately and subsequently combined. Costa et al. [6] measured the contribution of pig activity to dust concentration in a pig barn through the continuous monitoring of animal activity and confirmed the strong association that exists between animal activity and particulate matter concentration in animal houses. Also, Chung et al. [7] proposed an efficient data mining solution for the detection and recognition of pig wasting diseases using sound data in audio surveillance systems.

In this paper, we apply the computer vision technology to the daily-life activity monitoring of group-housed pigs in order to take care of pigs [8]. Caring for weaning pigs (21 or 28 days old) is the most important issue in livestock management because of their weak immunity. We aim for a real-time monitoring system for weaning pigs based on circadian rhythm. Furthermore, we should consider the practical issues such as cost in implementing the automated activity monitoring system, because poor profitability of pig farming has inhibited large-scale investment. In addition to this initial investment, managing individually-attached sensors such as accelerometers [9] in a large-scale pig farm may not be acceptable because of the managing cost. Thus, we consider a video camera, which does not need such managing overhead once installed, in order to monitor the group activity of pigs.

In a video-based monitoring system, we first extract activity data of weaning pigs from videos, and then investigate possible circadian rhythm from the activity pattern. It may be easy to detect a “specific” anomaly showing some sudden movements (i.e. the amount of activity caused by the appearance of a farm worker or a thief abruptly may be larger than the threshold determined by the abrupt normal movements). However, detecting an “unspecified” anomaly in the circadian rhythm (i.e. the total amount of activities in a pig’s room decreases/increases gradually due to some problems) is challenging. Increased or decreased overall movement within the group may imply an emerging disease [10] or change in behavior towards aggression due to a lack of feed or water. To the best of our knowledge, this is the first report on investigating the circadian rhythm of group-housed pigs under windowless and 24-hour light-on conditions by using visual data.

For a cost-effective solution [11, 12], we also analyze the tradeoff between the quality and the computational workload for implementing a practical system. A similar tradeoff issue has been widely studied in the video compression community recently [13-15]. The compression technique tries to maintain the required Peak Signal to Noise Ratio (PSNR), a quality metric widely used in the video compression community, with a minimum workload. In the activity monitoring applications for livestock, however, a reasonable metric for quality has not been reported. Thus, we first propose a quality metric (accuracy) which can be applied to the activity monitoring applications. Then, we can derive the minimum resolution size and frame rate for reasonable accuracy that is close to the baseline case with the maximum resolution size and frame rate.

This paper is organized as follows. Section 2 summarizes the related works, and Section 3 describes the proposed activity monitoring system. The details of the implementation and experimental results are explained in Section 4, and we provide some concluding remarks in Section 5.

 

2. Related Works

The last years, many research have been done to develop livestock monitoring systems. They can be classified in terms of the type of animal (i.e., pig, cow, and chicken) or sensor (i.e., a camera, a microphone, and attached sensors such as accelerometer, pedometer, RFID, etc.).

2.1 Pig Monitoring

2.2 Cow Monitoring

2.3 Chicken Monitoring

In summary, we can know that the automatic monitoring of the circadian rhythm of group-housed pigs by using camera data and the cost-effective solution of it have not been reported.

 

3. Pigsty Activity Monitoring System

3.1 Requirements of Activity Monitoring for Group-Housed Pigs

It is necessary to analyze the environmental characteristics of a pig farm in order to develop a practical monitoring system of group-housed pigs.

It is challenging to develop an automated monitoring system of group-housed pigs satisfying the above requirements. First of all, animals including pigs are known to be complex, individually different and time-variant (meaning that they respond differently at different moments of time) [1]. This contrasts with more classical approaches where pigs are considered as “an average of a population” due to its complexity as a steady state system.

Secondly, it is difficult to track many pigs in a room because of occlusion (as shown in Fig. 1), although it is possible to track a small number of pigs (3 pigs in a room) [16]. If an “individual” pig can be traced, a sick pig with less motion can be easily identified. However, for the reason explained in Section 3.2, we should check the activity on a “group” basis (based on the increased or decreased overall movement within the group), not an individual pig basis. Even manual monitoring cannot easily distinguish a motionless pig due to disease from a sleeping pig.

Fig. 1.Illustration of the occlusion caused by many pigs lying close together in a small room (picture taken from the Hamyang farm)

Lastly, we assume both food and water are supplied ad libitum. If a farm worker provides food manually, healthy pigs may show some motion at feeding time and it is easy to identify any pigs which are motionless at the feeding time. Also, we assume that the pig’s room is windowless and light is on continuously. Due to the ad libitum feeding and windowless/continuous lighting in the environment, there is no specific time when an anomaly can be detected easily.

3.2 Proposed Activity Monitoring System

In this paper, we focus only on the activity monitoring system. This system can be integrated with an existing control system that controls illumination, temperature, humidity, CO2, etc, and sends an alarm in an emergency. The goal of this research is not to replace but to support the farmers, who always remain a crucial factor in good animal management [1].

The overview of the activity monitoring system is shown in Fig. 2. The main idea of our system is to use the circadian rhythm. Many people believe every animal (including human beings) and plant on the earth has circadian rhythm [38, 39]. Although the amount of activity caused by a specific pig between 7 AM and 8 AM may be different every day, our hypothesis is that the total amount of daily activity caused by the pigs in a room may have some form of circadian rhythm. We design our activity monitoring system based on this circadian rhythm and verify our hypothesis with actual data.

Fig. 2.Overview of the activity monitoring system based on a video sensor

The procedures of the system are as follows. From a camera installed on the ceiling of a pig room, the 24-hour/365-day visual stream data are transmitted to a server through a LAN cable. Then, the server determines whether a given scene has any motion or not. If it has some motion, the server accumulates the motion information and computes the circadian rhythm. Finally, the server generates information on the circadian rhythm.

3.2.1 Motion Detection Module

The motion detection module requires a small computational workload for a low-cost solution and accuracy that can be applied to the activity monitoring applications. Thus, we compare the method using the frame difference [6] which has less computational workload with the Gaussian Mixture Model (GMM) method (background subtraction using background modeling) [40] which has high accuracy.

Fig. 3 shows the activity data of each method where the y-axis means the total amount of activity caused by the pig’s motion during one day. Note that the weaning pigs grow rapidly (500g/day for the first week, 600g/day for the second week, 700g/day for the third week, 800g/day for the fourth week, on average). Therefore, the total amount of activity should be increased gradually. Simple methods for extracting the daily-life activity such as the frame difference method may not guarantee the required accuracy. That is, the frame difference method is inappropriate for pig monitoring since the trend of pig’s growth cannot be found. Note that this is also true for the data obtained from another farm.

Fig. 3.Comparison of motion detection methods (data obtained from the Hamyang farm)

In order to provide the required accuracy, more complicated methods which have been widely used in video-based monitoring systems need to be considered, such as GMM. As shown in Fig. 3, the total amount of activity was increased gradually by using GMM. Note that this is also true for the data obtained from another farm. Furthermore, real-time processing of GMM is possible for a video stream by selecting a cost-effective resolution size and frame rate, although applying GMM straightforwardly to the 24-hour/365-day visual stream data generated from a large-scale pig farm may require too much implementation cost. This issue will be discussed in Section 3.2.3.

3.2.2 Circadian Rhythm Computation Module

The circadian rhythm computation module checks the circadian rhythm in order to detect any management problems. Also, the fact that the pigs grow every day during one month in the pigsty must be reflected in the pattern of the circadian rhythm.

We first determine whether the activity data has repeatability. Fig. 4 compares the values of the activity data accumulated at each hour. The activity data at 18 hour of June 9 was much smaller than that of June 10, so it is inappropriate to check the circadian rhythm by comparing the hourly data of today with that of yesterday.

Fig. 4.Repeatability of the activity data (data obtained from the Hamyang farm)

Fig. 5 shows the 24-hour pattern of activity data. Although it is difficult to derive a common pattern directly (the 24-hour patterns are inconsistent), we can see some form of the circadian rhythm (with low activity at night, even with the 24-hour light-on condition). In order to investigate the possible circadian rhythm further, Fig. 6 shows the frequency of the activity data measured for 26 days, where the x-axis means the total amount of activity accumulated during one hour and the y-axis means the frequency of the activity data. There is a clear trend of low activity at night-time (9 PM ~ 5 AM) and high activity at day-time (6 AM ~ 8 PM).

Fig. 5.Periodicity of the activity data (data obtained from the Hamyang farm)

Fig. 6.Frequency of the activity data, wehere the x-axis means the total amount of activity accumulated during one hour and the y-axis means the frequency of the activity data. (data obtained from the Hamyang farm)

Thus, we can exploit this circadian rhythm in monitoring pig activity. The activity data accumulated for 24 hours can provide enough consistency (as shown in Fig. 3(b)), and we can check the 24-hour data at the end of each day (12 PM). However, we also want to check the circadian rhythm more frequently in order to detect management problems as early as possible (which is the response time requirement discussed in Section 3.1).

To satisfy these conflicting requirements, we compare the 24-hour data of today with the 24-hour data of yesterday at each hour in order to reduce the inconsistency of hourly data and check the circadian rhythm more frequently. Fig. 7 shows the relationship between hourly data and 24-hour data. We also take into account the growth of weaning pigs [17] in checking the pattern of the circadian rhythm (the number of pixels corresponding to pigs from a top-view camera increases everyday by 2%).

Fig. 7.The relationship between hourly data and 24-hour data

For the purpose of explanation, we denote the hourly data AA(h) as the accumulated activity from h hour 0 minutes to h hour 59 minutes. The 24-hour data SAA(d, h) means the sum of the accumulated activity during 24 hours from h+1 hour of d-1 day to h hour of d day. As shown in Fig. 7, SAA(d-1, h) denotes the 24-hour data of yesterday at h hour, and SAA(d, h) denotes the 24-hour data of today at h hour. Therefore, we use SAA(d-1, h) and SAA(d, h) in order to compare the 24-hour data (circadian rhythm) at each hour and can detect any significant change in the circadian rhythm.

3.2.3 Cost-Effective Solution

The motion detection module is time-consuming, and there is a tradeoff between quality and computational workload required to obtain quality in the activity monitoring application [41]. We need to find the optimum tradeoff between the accuracy (the accuracy requirement discussed in Section 3.1) and the computational workload for applying to various activity monitoring applications. Once we find this optimum tradeoff, we can adjust the camera setting or downsample the input data.

For finding the optimum tradeoff, we first set the resolution size and frame rate to the maximum value supported by the camera (called the base-case), and then compute the hourly data obtained from the activity monitoring data. After setting the resolution size and frame rate to each downsampled value (called the downsampled-case), we compute another hourly data obtained from the downsampled activity monitoring data. Then, we compute the similarity between the base-case and the downsampled-case.

Generally, the correlation of two random variables is a standard measure of how strongly two variables are linearly related. Correlation therefore naturally captures our intuitive notion of temporal similarity. The temporal similarity between the two cases (the base-case and the downsampled-case) can be computed by

Let xb,i be hourly data obtained from the base-case at each hour i and xd,i be hourly data obtained from the downsampled-case at each hour i. Note that hourly data means the accumulated amount of activities detected by GMM during an hour. For the base-case b, μ(xb) and σ(xb) denote the mean and the standard deviation, respectively. For a particular downsampled-case d, μ(xd) and σ(xd) denote the mean and the standard deviation, respectively. Using equation (1), we can compute the similarity (i.e. accuracy), and finally compute the relative tradeoff (=accuracy/workload) with various resolution/frame rate cases.

Furthermore, we can minimize the resource requirements of the memory/CPU in the circadian rhythm computation module by employing incremental updating on activity data in a circular array. As shown in Fig. 8, we first prepare an array CA of size 48 as a circular array in order to store the hourly data. Then, at every hour, we increment t and store the hourly data of t into CA[t mod 48]. Then, the sum of CAs from [(t-23) mod 48] to [t mod 48] means the 24-hour data of today, whereas the sum of CAs from [(t-24) mod 48] to [(t-47) mod 48] means the 24-hour data of yesterday. The 24-hour data can be updated easily as

Fig. 8.Circular array CA for minimizing the memory/CPU requirement

 

4. Experimental Results

In this section, we present the experimental results to demonstrate the performance of the proposed real-time monitoring system. The real-time monitoring system comprised a video sensor and a server. As shown in Fig. 9, we installed a camera at the ceiling of a pig room and set the resolution size to 1280×720 pixels and the frame rate to 30 frames/second (fps) initially. With this initial setting, we acquired two datasets in two different environments (22 weaning-pig data was obtained from the Hamyang farm in June, and 24 weaning-pig data was obtained from the Jinju farm in March) and measured the amounts of activities caused by pigs in each pigsty environment separately. The experiments were performed on an Intel Core i5-2500 at 3.3GHz 4-core processor with 4GB of RAM.

Fig. 9.Picture of a pig house with a video sensor installed (picture taken from the Hamyang farm)

4.1 Motion Detection

As explained in Section 3.2.1, the motion detection module requires a small computational workload for a low-cost solution and accuracy that can be applied to the activity monitoring applications. Thus, we compared the method using the frame difference with the GMM [40].

Fig. 10 shows the output results of the two methods. Although the frame difference method is very simple, the output shows some false positive and negative errors. The output of the GMM method also has some errors, but the method can provide the required accuracy by detecting the two moving pigs. Section 4.3 will discuss how the computational workload of the GMM method can be reduced.

Fig. 10.Comparison of motion detection results (data obtained from the Hamyang farm)

4.2 Circadian Rhythm

One of the goals of the monitoring system is to investigate the circadian rhythm of group-housed pigs under windowless and 24-hour light-on conditions. Fig. 11 shows various 24-hour data computed at 0 AM, 8 AM, and 4 PM (although we show three 24-hour data only, all 24-hour data show similar patterns.). Unlike the frame difference method explained in Section 2.2.1, the total amount of activity was increased gradually by using GMM with various h values. Also, at every hour h, the 24-hour data of today SAA(d, h) is in the range of (85+2)% to (115+2)% of the 24-hour data of yesterday SAA(d-1, h), where 2% in the range corresponds to the pig’s growth. That is, the 24-hour data of today is similar to that of yesterday, at every hour.

Fig. 11.Illustration of the circadian rhythm at 0 AM, 8 AM, and 4 PM (data obtained from the Hamyang farm)

As explained in Section 3.1, it may be difficult to predict the group activity of pigs at any time accurately. Because of the circadian rhythm, however, we believe it is possible to determine whether the 24-hour data is different from yesterday (in regard to the circadian rhythm changes possibly caused by disease or thermal discomfort) “as early as possible” (at every hour), without complicated learning techniques or any training process.

To investigate the circadian rhythm in a variety of pigsty environments, we used another dataset acquired from a different pigsty environment. Fig. 12 shows the circadian rhythm from the Jinju farm, and we can also see consistent 24-hour data across days (although we show one 24-hour data only, all 24-hour data show a similar pattern).

Fig. 12.Illustration of the circadian rhythm at 0 AM (data obtained from the Jinju farm)

4.3 Cost-effective Solution

Another goal of the monitoring system is to implement the system in a cost-effective way. We set the resolution size to 1280×720 pixels and the frame rate to 30 frames/second (fps) initially (base-case) to derive the optimal resolution size and the optimal frame rate with reasonable accuracy. From the base-case of 1280×720 pixels and 30 fps, we could obtain various downsampled resolutions of 640×480, 320×240, 160×120, and 80×60 pixels and downsampled frame rates 15, 10, 5, and 1 fps. Fig. 13 shows some examples of the hourly data with various resolution/frame rate cases. Some cases have similar patterns (320×240 pixels/10 fps vs. 320×240 pixels/1 fps), whereas other cases have entirely different patterns (320×240 pixels/10 fps vs. 80×60 pixels/10 fps). Thus, we should derive the downsampled-case whose hourly data is similar to that of the base-case with minimal computational workload.

Fig. 13.The hourly data with various resolution/frame rate cases (data obtained from the Hamyang farm)

For describing the temporal similarity between the base-case and the downsampled-case, we computed the similarity using Equation (1). Table 1 shows the similarity values normalized to the base-case (i.e. the similarity of the base-case itself is 1). Note that the similarity is more affected by resolution than frame rate. For describing the workload, we measured the execution time. Table 2 shows the relative execution time normalized to the base-case (i.e. the relative execution time of the base-case itself is 1). Finally, we represent the relative value of “accuracy/workload” tradeoff in Table 3 (i.e. the relative tradeoff of the base-case itself is 1). Therefore, we can derive the cost-effective resolution size and frame rate as 160×120 pixels and 1 fps, with acceptable accuracy (i.e. in this paper, we consider the similarity of 0.9 as the acceptable accuracy). To the best of our knowledge, this is the first report on the tradeoff in monitoring the continuous and large incoming data stream that is characteristic of monitoring systems (i.e. with 1800 frames obtained from a camera, we could reduce the running time of 86.4 seconds to 2.6 seconds without degrading the accuracy significantly).

Table 1.Similarity (i.e., accuracy) with various resolution/frame rate cases

Table 2.Relative execution time (i.e., workload) with various resolution/frame rate cases

Table 3.Relative tradeoff (=accuracy/workload) with various resolution/frame rate cases

 

5. Conclusion

The automated activity monitoring of livestock is an important issue in large-scale livestock management. We proposed a real-time pigsty monitoring system and cost-effective technique for analyzing weaning pig’s activities using visual information acquired from a camera installed in the pig house. Especially, this research focused on the circadian rhythm of group-housed pigs from the 24-hour/365-day visual stream data, and the practical issues such as the quality-workload tradeoff in implementing a pig activity monitoring system.

From the experiments, the circadian rhythm can be found in group-housed pigs under the windowless and 24-hour light-on conditions, and a monitoring system can be made based on the circadian rhythm. We also found that our method for deriving the cost-effective solution can satisfy the low cost requirement without degrading the accuracy significantly.

As a future work, we have a plan to implement the proposed solution with an embedded system. Also, we have a plan to implement the object tracking technique such that individual pig can be identified.