A shared branch prediction mechanism is provided in which a pool of branch prediction storage locations are shared among the multiple cache lines comprising a row of the instruction cache. The branch prediction storage locations within the pool are dynamically redistributed among the cache lines according to the number of branch instructions within each cache line. A cache line having a large number of branch instructions may be allocated more branch prediction storage locations than a cache line having fewer branch instructions. A prediction selector is included for each cache line in the instruction cache. The prediction selector indicates the selection of one or more branch prediction storage locations which store branch predictions corresponding to the cache line. In one embodiment, the prediction selector comprises multiple branch selectors. One branch selector is associated with each byte in the cache line, and identifies the branch prediction storage location storing the relevant branch prediction for that byte. In another embodiment, each set of two bytes within a cache line shares a portion of the pool with the corresponding set of two bytes from the other cache lines within the pool. The prediction selector for the cache line indicates which sections of the cache line have associated branch prediction storage locations allocated to them, as well as a taken/not-taken prediction associated therewith. The first taken prediction within the line subsequent to the offset indicated by the fetch address is the branch prediction selected.