This paper addresses the problem of cooperation between learning situated agents. We present an agent’s architecture based on a satisfaction measure that ensures altruistic behaviors in the system. Initially these cooperative behaviors are obtained by reaction to local signals emitted by the agents following their satisfaction. Then, we introduce into this architecture a reinforcement learning module in order to improve individual and collective behaviors. The satisfaction model and the local signals are used to define a compact representation of agents’ interactions and to compute the rewards of the behaviors. Thus agents learn to select behaviors that are well adapted to their neighbor’s activities. Finally, simulations of heterogeneous robots working on a foraging problem demonstrate the interest of the approach.