In the discussion about consequences of the release of genetically modified (GM) crops, the meaning of the term “environmental damage” is difficult to pin down. We discuss some established concepts and criteria for understanding and evaluating such damages. Focusing on the concepts of familiarity, biological integrity, and ecosystem health, we argue that, for the most part, these concepts are highly ambiguous. While environmental damage is mostly understood as significant adverse effects on conservation resources, these concepts may not relate directly to effects on tangible natural resources but rather to parameters of land use or ecological processes (e.g., the concept of biological integrity). We stress the importance of disclosing the normative assumptions underlying damage concepts and procedures for the evaluation of damages by GM crops. A conceptualization of environmental damage should precede its operationalization. We recommend an unambiguous definition for damage developed earlier and recommend that evaluation criteria be based on this. However, a general damage definition cannot replace case-specific operationalization of damage, which remains an important future challenge.