We call this feast "Peter AND Paul", which is in juxtaposition to the "Peter OR Paul" or "Peter VS. Paul" thinking that often goes on.
Though there may have been some amount of it at the time, we see now that theirs is not a rivalry but a complementary relationship. Many of our brothers and sisters in Christ, throughout time, have tried to emphasis one over the other, or to surpass one with the other in order to explain some point or to justify/deny some belief system or tradition. But regardless of how they lived and ministered, they died in sync in Rome. One a citizen, the other a Jew. One martyred 'nobly' the other ignobly but in the end, both martyred for Christ.
So it is Peter AND Paul, as apostles of Jesus, that we celebrate; they who did Christ's bidding, not for their own gain but for his. Neither surpasses the other in preaching or theology and most importantly, neither surpasses Christ. We celebrate them together like so many of the early Roman martyrs (tomorrow for example).
I think that is what so many people miss. We do not "follow Paul or Apollos" (cf. 1 Corinthians), we follow Jesus with their help and guidance. It is his mandates and path, not Peter's or Paul's that we follow. So many problems between denominations would be solved if we just walked that line.
It is truly right and just, our duty and our salvation,
always and everywhere to give you thanks,
Lord, holy Father, almighty and eternal God.
For by your providence
the blessed Apostles Peter and Paul bring us joy:
Peter, foremost in confessing the faith,
Paul, its outstanding preacher,
Peter, who established the early Church from the remnant of Israel,
Paul, master and teacher of the Gentiles that you call.
And so, each in a different way
gathered together the one family of Christ;
and revered together throughout the world,
they share one Martyr’s crown.
And therefore, with all the Angels and Saints,
we praise you, as without end we acclaim:
Holy, Holy, Holy Lord God of hosts...
-- Preface from the Solemnity of Peter and Paul