The problem of power allocation for the secondary user (SU) in a cognitive radio (CR) network is investigated in this paper. The primary user (PU) is protected by the average interference power constraint. Besides the average interference power constraint at the PU, the transmit power of the SU is also subject to the peak or average transmit power constraint. The aim is to balance between the goal of maximizing the ergodic capacity and the goal of minimizing the outage probability of the SU. Power allocation schemes are then proposed under the aforementioned setups. It is shown that the proposed power allocation schemes can achieve high ergodic capacity while maintaining low outage probability, whereas existing schemes achieve either high ergodic capacity with high outage probability or low outage probability with low ergodic capacity.