2.1.

Study Area and Dataset

During the 2015 growing season, four corn experiments were established in 1.2 hectares at Ashland Bottoms Farm, Manhattan, Kansas (39.13°N, $-96.6°\mathrm{E}$, 314 m above sea level) (Fig. 1).

Fig. 1

(a) Study area with four corn experiments evaluating: (i) hybrids, (ii) fertilizer $N$ rates, (iii) plant densities, and (iv) plant density gaps and (b) photo of the UAS S800 DJI hexacopter mounted with RGB sensor.

The nitrogen experiment (NE) was implemented in $9.1\mathrm{m}\times 10.6\mathrm{m}$ plots, with five N fertilization levels using urea ammonium-nitrate ranging from 0 to $200\mathrm{kg}\mathrm{N}{\mathrm{ha}}^{-1}$, in $50\mathrm{kg}\mathrm{N}{\mathrm{ha}}^{-1}$ intervals. The plant density experiment (PE) and plant density gap experiment (PGE) were conducted in $6.1\mathrm{m}\times 10.6\mathrm{m}$ plots, whereas the hybrid experiment (HE) was planted in $6.1\mathrm{m}\times 12.2\mathrm{m}$ plots. All experiments were evaluated in randomized complete block design with five replications. Across all studies, row spacing was 0.76 m. Target plant density was $8.4\text{plants}{\mathrm{m}}^{-2}$ in NE and HE, and a range between 4.4 and $10.4\text{plants}{\mathrm{m}}^{-2}$ for the PE and PGE. Corn hybrid used in NE, PE, and PGE was DK61-88 (Dekalb^®, Monsanto) 111 days commercial relative maturity (CRM). For the HE, corn hybrids evaluated were DK61-88, DK63-55 (113 CRM), DK64-69 (114 CRM), and P1105 and P1151 (111 CRM; Dupont Pioneer^®). All four corn trials were used as a base line to generate spatio-temporal variability of plant height, biomass, and yield to evaluate UAS and structure from motion under different plant height scenarios.

2.2.

Platform, Sensor, and Ground-Truthing

An UAS platform (S800, DJI, Shenzhen, China) was used to collect aerial imagery. This platform includes the Wookong-M onboard autopilot system and GPS v2 unit (S800, DJI, Shenzhen, China). Flight missions and parameter settings were assigned using UgCS ground station software.³³ The platform sensor included in each flight was Alpha ILCE A5100 RGB Sony (Tokyo, Japan), mounted with a Sony SELP1650 PZ 16-50 mm lens (sensor resolution is $6000\times 4000\text{pixels}$). Aperture and exposure time were adjusted manually prior to each flight mission considering the ground speed of the UAS and light conditions at the time of flights. In both flights, camera setting was performed using manual exposure control; shutter speed was set to $1/4000\mathrm{s}$, aperture to f5, and ISO to 640 and 16 mm focal length configuration. Two UAS missions were performed (17 and 29 July). Highly visible yellow and black ($1\mathrm{m}\times 1\mathrm{m}$) cross-centered plastic ground targets were used as ground control points (GCPs). In this project, 14 yellow and black cross-centered plastic ($1\mathrm{m}\times 1\mathrm{m}$) GCPs were used as main sources for imagery geolocation. The GCPs were distributed on the borders and internal alleys of the experiments following³⁴ the recommendations. The average distance between GCPs was 42 m in both missions.

Two critical corn growth stages were identified as target candidates for UAS missions: (1) the late vegetative herein termed as preflowering and (2) onset of reproductive or flowering stage.³⁵ These UAS mission timings were relevant because of the importance of the aforementioned corn growth stages to determine if plant height estimates can populate crop yield forecasting models. The goal of this step was to overlay CSMs from UAS with ground-truthing data then check goodness-of-fit of CSMs to capture spatio-temporal change of plant height at both stages. The ground-based data collection was divided into destructive biomass sampling and nondestructive in situ plant height measurements. GCPs and plant samples were georeferenced by implementing a Global Navigation Satellite System–Real-Time Kinematic survey for spatial and temporal monitoring. The data layer containing the geolocated plant positions was overlaid with the orthomosaic and CSMs using ArcMap (ArcGIS v10.3, Environmental System Research Institute Inc.).³⁶ Absolute plant height, field ground-truthing, was measured via a centimeter resolution wooden ruler. Field sampling procedures define absolute plant height as the vertical distance between the base of stem and the top region of the plant where leaves reach maximum height without any external intervention ($n=331$ plants measured 2 weeks before flowering and $n=331$ plants determined at flowering). Stem diameter ($n=331$ measured) was determined at the base of the plant following the procedure described by Ciampitti et al.²⁰ The field measurement performed 2 weeks before flowering was separated by 5 days from the UAS mission; thus plant height was adjusted to the date of the UAS mission using the observed plant height change rate computed between flowering and 2-weeks prior. This adjustment did not modify the proportion of variation accounted for the aerial imagery but significantly reduced the bias in the final observed plant height values, with lower plant height values for the ground-truth data (adjusted by 5 days within the period of height growth). For biomass determination, each individual plant was cut at the stem base and fresh weight was collected in situ. Both stem diameter and plant biomass were measured only at flowering time.

2.3.

Data Processing Workflow: Crop Surface Model, Orthomosaic Generation, and Plant Height

The UAS missions were conducted at 65-m altitude to achieve a ground sampling distance, expressed as the distance between the centers of two consecutive pixels measured on the ground, of 0.015 m. An overlapping and side lapping of 80% was employed in accordance with Photoscan manual recommendations for successful CSM reconstructions [Fig. 2(a)].³⁷ Ground speed setting of the UAS was $7\mathrm{m}{\mathrm{s}}^{-1}$ obtaining one image per 1.8 s to achieve the expected overlapping on the track of the UAS. A total of 265 images were collected per mission. The GCPs and UAS imagery data set were integrated and processed for true color [red, green, and blue (RGB)] orthomosaic and CSMs for plant height.

Fig. 2

(a) Workflow data integration between UAS and Photoscan and (b) Photoscan–ENVI–ArcMap data workflow.

A workflow for CSMs reconstruction was implemented using Photoscan [Fig. 2(a)]. Processing steps included: feature matching, solving camera intrinsic, and extrinsic orientation parameters, reconstructing of the dense point cloud (DPC), and texture mapping. Parameter setting for imagery alignment presented the following characteristics: low accuracy and referenced pair preselection, tie and key points limited to 0 and 40000. The Photoscan imagery aligment algorithm detects points in the source images, which are stable under changing viewpoints and lighting conditions.³⁷ Then, Photoscan software generates a descriptor for each point based on its local neighborhood. These descriptors are used to detect correspondences across the images.³⁷ Later the software estimates the camera intrinsic and extrinsic orientation parameters using the internal bundle-adjustment algorithm to approximate accurate camera locations.³⁷ The distance between all GCPs was comparable and located along the image data set to minimize horizontal and vertical geometrical error. The DPC was reconstructed by Photoscan by implementing the height-field algorithm based on pair-wise depth map computation. Moreover, the quality value for the DPC recontruction was set to medium for optimizing the computation time and data set size following Photoscan manual recomendations. The DPC reconstruction achieved 2550 and $2765\text{points}/{\mathrm{m}}^2$, respectively, for each mission timing. A spatial interpolation procedure, via inverse distance weighting (IDW), was applied to the DPC to generate the CSM. Orthomosaic and CSM native Photoscan spatial resolutions were 1.0 and $2.0\mathrm{cm}/\text{pixel}$ for data sets from both missions.

The absolute plant height estimation was solved as the difference between the CSM and a digital terrain model (DTM) of bare ground surface [Fig. 2(b)]. The DTM was reconstructed from the flowering RGB orthomosaic (captured 2-weeks prior) and CSM data sets.²⁶ The first step includes the ground class segmentation in the RGB orthomosaic [Fig. 2(b)]. A support vector machine (SVM) classification was implemented in ENVI³⁸ to solve the ground class segmentation [Fig. 2(b)]. The training data set included 4000 pixels and iterated in “vegetation,” “bare soil,” and “shadow” classes. In the iteration phase, a linear discriminant was explored with unsatisfactory results ($\text{over all accuracy}=0.55$). Thus, a nonlinear classification approach was implemented on the decision surface hyperplane and a radial kernel function was utilized for discrimination between classes. The gamma in kernel function was set to 0.25, the inverse of the number of computed attributes,^39,40 and the penalty parameter was set to 95.⁴⁰ The overall accuracy of the nonlinear SVM classification on the three classes was 0.79. The “bare soil” raster was exported from the CSM with bare soil areas into ArcMap and the DTM solved by using the overlapping CSM vertical and horizontal determined from the bare soil over the bare soil class data from segmented ground class regions. Ground class regions utilized in the IDW interpolation included the borders and alleys of the trials, considering an average distance of 12 m between adjacent alleys.

The absolute plant height estimated data were obtained by a map algebra subtraction between the CSM and the DTM over 0.08-m cylinder radius length assigned to each plant center location [Fig. 2(b)]. Estimated plant height was assigned to upper mean quintile CSM pixels in the cylinder area assigned to each plant.

2.4.

Plant Height Validation and its Relationship with Stem Volume and Biomass

Plant height data extracted from UAS imagery analysis and collected from field measurements (ground-truth data) were linearly regressed using the GraphPad Prism software.⁴¹ The proportion of variation accounted by the fitted model at each developmental stage was evaluated. In addition, a linear relationship between plant biomass and stem volume calculation was examined; whereas an exponential model was fitted for the plant biomass and plant height obtained via CSM. Both fits were performed using the GraphPad Prism software. For plant height validation, model fit was calculated by determination of the root mean square error (RMSE, measurement of estimated versus observed values). Outlier detection was executed via the robust standard deviation of the residuals.⁴²

An allometric evaluation was performed for plant height data extracted from UAS imagery and within-field measurements. Thus, reduced major axis was performed with the Standardised Major Axis Estimation and Testing Routines (SMATR) contributed package⁴³ to R development software.⁴⁴ For the different phenological timings, slopes were tested to compare independent fit versus a shared fit for this parameter (if slopes are equal or not). Parameters were ${\log }_{10}$ transformed ($Y=\alpha X^{\beta }\to \log Y=\log \alpha +\beta \log X$) prior to the analysis⁴⁵ and normal distribution of residuals was verified.

3.1.

Crop Surface Model and Orthomosaic Generation

Early process of CSM construction is presented in Fig. 3. The UAS images taken at different sections were stitched together using GCPs as a reference (Fig. 3) in PhotoScan software. The importance and the number of GCPs necessary to ensure accuracy of UAS image construction have been previously discussed by other researchers.^46–48

Fig. 3

CSMs for estimated absolute plant height on top of the corn canopy: (a) 2-weeks prior to flowering and (b) flowering time. Upper part: 3-D view and; lower part: 2-D perspectives for corn plant height. Note: The blue color represents ground and low vegetation, the yellow refers to short-medium corn plants, and the brown and red colors represent taller plants within the corn canopy.

In terms of geometric quality, the accumulated horizontal and vertical error was $0.7\mathrm{cm}/\text{pixel}$ in orthomosaic and CSM from 2-weeks prior to flowering, and $0.5\mathrm{cm}/\text{pixel}$ for the flowering raster products. Furthermore, a woody table ($\text{dimensions}=0.8\mathrm{m}\text{length}\times 0.4\mathrm{m}\text{wide}\times 0.6\mathrm{m}\text{height}$) was used for nonvegetation geometric evaluation. A total of six local GCPs were implemented along the top of the table to evaluate the vertical displacement between the original GCPs and the same one located in the CSM reconstruction. The vertical error in this case was $0.6\mathrm{cm}/\text{pixel}$.

3.2.

Plant Height from Unmanned Aerial Systems Versus Ground-Truth Plant Trait

A strong positive correlation was obtained between CSM-estimated plant height and ground-truth data collected when corn plants were at flowering stage ($R^2=0.79$, $\mathrm{RMSE}=0.09\mathrm{m}$, $n=331$, and $\text{mean}=1.84\mathrm{m}$) [Fig. 4(b)]. The correlation between CSM and ground-truth data measured two weeks prior to flowering ($R^2=0.63$, $\mathrm{RMSE}=0.11\mathrm{m}$, $n=331$, and $\text{mean}=1.05\mathrm{m}$) was relatively weaker (lower $R^2$) and with a slightly higher RMSE [Fig. 4(a)]. The RMSE to mean plant height ratio prior to flowering was 14%, close to threefold higher compared to the ratio estimated at flowering time (5%). For the preflowering measurement, the lower proportion of the variation accounted for the CSM-estimated plant height was primarily due to lack of uniform development within the corn canopy and plants emerging at different timing due to soil–weather factors (e.g., saturated soil areas, low residue with less temperature). At flowering, maximum plant height was attained,⁴⁹ corn canopy become more uniform with less heterogeneity (lower RMSE to mean plant height ratio) and better prediction power (higher $R^2$).

Fig. 4

Plant height estimation via UAS imagery collection (a) 2-weeks prior, (b) at flowering, and (c) log–log linear regression of estimated- to observed-plant height (determined from the ground base to the top of the canopy). RMSE, root mean square error.

A correlation obtained between measured and CSM-estimated plant height is consistent with previous findings for corn,^25,26,50 barley (Hordeum vulgare L.), and rice (Oryza sativa L.).^27,51 A significant correlation between plant height measurements at flowering stage support the conclusion drawn by Geipel et al.⁵⁰ that imagery taken at end of stem elongation is better correlated with ground-truth data. Few researchers have studied the corn growth stage that UAS imagery should be taken to improve plant height estimation.^50,52,53 Other studies not using UAS imagery to evaluate the relationship between actual plant height and remotely sensed plant height⁵² also concluded that late vegetative stage sensor-based plant height measurements correlated with actual plant height. Additionally, plant height measurements at late stage of corn were found to correlate with grain yield.⁵⁴

It is worth noting that at both corn stages plant height was underestimated, similar to the findings presented by Grenzdörffer²⁶ and Shi et al.⁵³ For understanding the stability of the plant height estimation, two evaluations were executed. The first one was done by comparing the linear regression slopes (for equality) of the estimated- and observed-plant height relationship [Figs. 4(a) and 4(b)] between the two growth stages evaluated to understand the stability of the estimation between dates and across plant height class (log–log transformation analysis) [Fig. 4(c)]. Results showed similar slopes across classes for both mission timings. A second evaluation was performed to understand the absolute and relative magnitude of the plant height estimation for both dates. Ground-truth plant height data were divided into three equal classes for each crop stage. Prior flowering ground-truth mean plant height data classes were low 1.22 m, medium 1.52 m, and high 1.71 m. For the 2 weeks before flowering timing, within the low plant height group, 22% of the data in this class were underestimated by the CSM-estimated plant height trait; while for the high plant height group, this analysis resulted in 24% of the plant height observations being underestimated. At flowering, only 10% of the data on plant height across all classes (low 1.77 m, medium 2.12 m, and high 2.24 m) were underestimated by the CSM-estimated plant height trait. Synthesizing, this analysis allowed us to conclude that there was a better prediction of plant height due to a lower underestimation at flowering, which was also related to lower plant heterogeneity within the corn canopy.

3.3.

Unmanned Aerial Systems Based Plant Height Relation with Biomass

Since ground-truth plant height was better estimated at flowering, the biomass data collected at the same growth stage were utilized to better understand the relationship between plant height and plant biomass. Plant biomass and CSM-estimated plant height exhibited a statistically significant correlation at flowering [Fig. 5(a)]. However for the plant biomass trait, the proportion of the variation accounted by the CSM-estimated parameter alone was low ($R^2=0.31$, $n=332$, and $P<0.05$). Examination of Fig. 5(a) shows substantial variation present in the data, and possibly nonlinear behavior at greater plant height values. Plant biomass estimation substantially improved ($R^2=0.79$, $n=332$, and $P<0.05$) when the stem diameter (determined at equal growth stage and for the same plants as the plant height trait) was considered as a part of the stem volume calculation [Fig. 5(b)]. Allometric equations were previously utilized in corn to predict biomass with different levels of success depending on the variation of the data (genotype by environment by management interaction) and the timing of the sampling.^15,55,56 More accurate biomass estimation performed via utilization of allometric models could be utilized as a tool to forecast corn yields. Last, improvements of biomass prediction for corn after flowering stage were documented when the apical ear shoot diameter (maximum diameter of the ear organ) was included in the stem volume calculation.²³ Thus, improvement in corn biomass prediction will be of a great challenge for the remote sensing discipline because the reproductive organs (ears) are placed at varying positions within the corn canopy. In a simplified approach, a combination of various data layers collected from multiple sensors [e.g., plant height, stand counts, normalized difference vegetation index (NDVI)⁵⁷] in a spatio-temporal fashion might allow to adjust in real-time corn yield estimations based not only on plant size but also considering plant nutrient status and the complex interaction with the environment.

Fig. 5

Per-plant biomass (dry basis, expressed in g ${\mathrm{pl}}^{-1}$) versus (a) plant height trait estimated via CSM and (b) stem volume estimated via implementation of the volumetric cylinder equation (including plant height estimated via CSM, CSM-estimated plant height) all parameters determined at flowering.

For corn crop, a correlation between ground-truth plant height measured at late vegetative or early reproductive and plant biomass has been previously documented.^12,15 For the current study, plant height was adequately predicted for both corn growth stages: 2-weeks before (with more variation detected) and at flowering. Similarly, a significant positive relationship between corn plant height measured late-vegetative using sensors, mounted on satellite or run manually, with biomass or yield was reported.^17,58 Not many UAS based results are available for corn,^50,53 but our results on the relationship between plant height measured by UAS platform with biomass is in agreement with previous research reported for other field crops.^27,51

Last, the relationship documented in this study for corn crops between stem diameter and plant biomass is in line with findings previously presented by Mourtzinis et al.¹⁵ A nondestructive way of measuring stem diameter from images mounted on UAS and other ground-truth sensors remains as a critical research gap for improving plant biomass prediction and the potential for yield forecasting purposes. From a remote sensing standpoint, different vegetation indices and multi/hyperspectral sensors can be investigated to improve plant biomass prediction and yield forecast procedure.