In this article we present a complete control framework which aims at making humanoid robots capable of carrying objects together with humans. Firstly, we review collaborative carrying by creating a human-inspired taxonomy. From this taxonomy, we identify the required primitive subtasks. Next, these are formulated into constrained optimization problems for controlling the whole-body motion of a humanoid robot. The subtasks include two walking pattern generators that account for physical collaboration, as well as posture and grasping controllers. Finally, we validate our framework in a variety of collaborative human-robot carrying experiments, using HRP-4.