In the regime of strong light-matter coupling, polariton modes are formed that are hybrid light-matter excitations sharing properties of both, an electrodynamic cavity mode and a matter mode. Recently, magnon-polaritons are intensively researched in ferromagnetic materials in the microwave range, with potential applications for quantum computing and sensors. In the recent decade, polaritons were obtained in the THz range with various excitations like, intersubband plasmons, magnetoplasmons in two-dimensional electron gases or vibrational modes of molecules. However, exploring magnetic excitations instead of dielectric transitions gives an advantage of low damping of spins. We are investigating coupling of the antiferromagnetic resonance (AFMR) with THz cavity modes. Here, we report on cavity-mediated magnon-magnon coupling in a system consisting of two parallel-plane crystals forming a Fabry-Perot type cavity. A crystal of yttrium ferrite (YFeO3) is kept at room temperature, while a crystal of hematite (alpha-Fe2O3) is fixed on a copper mirror placed on a heater. Spin dynamics in both materials are characterized by low damping. We used a monochromatic continuous-wave spectrometer operating in the range of 0.2-0.35 THz, which is based on frequency extenders to a vector network analyser. Reflection spectra measured as a function of hematite temperature show a series of cavity modes that form avoided crossings with the AFMR in the hematite crystal, frequency of which is rising with temperature. By measuring temperature-differential spectra, we reveal only cavity modes that are coupled to the AFMR in hematite. That is because the AFMR in the YFeO3 crystal does not depend on temperature of the hematite crystal, therefore modes coupled to YFeO3 do not show up in temperature-differential spectra. Contrary, differential spectra to external magnetic field reveal only cavity modes coupled to the AFMR in YFeO3 that has a frequency of about 300 GHz. Since the magnetic field is applied in a direction that does not change the AFMR in hematite, magnetic field-differential spectra do not show the modes coupled to the AFMR in hematite. Differential to a gap between the two crystals reveals the cavity modes. Under certain gap between the crystals, we can observe cavity modes that are strongly coupled to the AFMR in both crystals at the same time, which suggest that magnons in both crystal are coupled via a cavity mode.