Time your pepper harvest based on the type of pepper and desired flavor.
Sweet green bell peppers are usually harvested when full size, firm and still green. If left on the plant, they will color up and have a sweeter flavor. Colorful bell and other sweet pepper varieties can be harvested at the green stage or once they develop their red, orange, yellow, purple, or brown color.
Most hot peppers are harvested when red, fully mature and have the strongest flavor. Green chiles and jalapenos are harvested when green. And some like jalapenos, habanero and Anaheim are hot whether harvested when green or red.
Cut, do not pull the fruit off the plants to avoid damage.
And label hot peppers in the garden, during harvest, and preparation. Many sweet and hot pepper varieties look similar and confusing the two could result in an unwelcome surprise.
A bit more information: The white membrane – not just the seeds – provide the heat in hot peppers. Removing this helps tame the flavor. Keep your hands away from your eyes when working with hot peppers. Wash your hands, cutting boards and utensils thoroughly when done.