その間の長寛3年（1163年）、興福寺・延暦寺の抗争事件に際して、議定の場で親平氏の延暦寺を支持したことで興福寺の怒りを買い、放氏される（興福寺の強訴の一環として、氏寺・氏社に不利益をもたらした氏人の追放を興福寺別当から氏長者に通告する、追放が解除されない限り朝廷に出仕できない）、異母弟の藤原成親（母は藤原経忠の女）が後白河の平氏打倒計画に参加して処刑される（鹿ケ谷の陰謀）などの政治的危機もあったが、平氏との友好関係を維持した隆季の立場が揺らぐことはなかった。

During that time, in 1163, he was in political peril, such as his banishment from the clan (as a part of the direct petition by Kofuku-ji Temple, Kofuku-ji betto (the head priest of Kofuku-ji Temple) notified the uji no choja (the head of the clan) of the banishing of the clan member who disadvantaged the Uji-dera Temple (temple built for praying clan's glory) and Uji-sha Shrine (shrine built for praying clan's glory), and unless the banishment was rescinded the banished clan member was not allowed to serve in the Imperial Court and the execution of his youger step brother, FUJIWARA no Narichika (his mother was a daughter of FUJIWARA no Tsunetada), for participating in Emperor Goshirakawa's scheme to bring down the Taira clan (the Shikagatani Plot), because he angered Kofuku-ji Temple for supporting the pro-Taira clan Enryaku-ji Temple at gijo no ba (the site where an agreement was to take place) during Kofuku-ji Temple and Enryaku-ji Temple Feuding Incident, but Takasue, who maintained an amicable relationship with the Taira clan, stood firmly on his ground. - Wikipedia日英京都関連文書対訳コーパス