Living organisms are characterized by their smooth adaptability to environmental changes and their robustness against morphological modifications. To investigate the computational mechanisms behind such learning scheme, we proposed tacit learning as a novel learning method. In tacit learning, there are no clear distinctions between learning and motor control: learning is a simple accumulation process embedded in the controller. In previous work, tacit learning was applied with success to bipedal locomotion of a 36 DoF humanoid robot. In this paper, we generalize the structure of the controller such as applying adaptive integration to a wider range of systems and behaviors. This is achieved by applying the principle of tacit learning in a hierarchical fashion, in which the value of a virtual periodic dynamic variable is tuned for continuous adaptation. This resulting PD-PI (proportional-derivative periodic-integration) controller preserves the advantages of tacit learning that the controllers do not include any prior knowledge of the system in which they are embedded. It also shares with biological systems the property that control and adaptation progress without explicit distinction between them.