An important question in designing cryptographic functions including substitution boxes (S-boxes) is the relationships among the various nonlinearity criteria each of which indicates the strength or weakness of a cryptographic function against a particular type of cryptanalytic attacks. In this paper we reveal, for the first time, interesting connections among the strict avalanche characteristics, differential characteristics, linear structures and nonlinearity of quadratic S-boxes. In addition, we show that our proof techniques allow us to treat in a unified fashion all quadratic permutations, regardless of the underlying construction methods. This greatly simplifies the proofs for a number of known results on nonlinearity characteristics of quadratic permutations. As a by-product, we obtain a negative answer to an open problem regarding the existence of differentially 2-uniform quadratic permutations on an even dimensional vector space.