A web scraper that collects notices from the Institute of Science and Technology, Tribhuvan University (IOST TU) website and serves them through a clean FastAPI backend with a Next.js frontend.

The scraper walks the IOST notices page, extracts each notice’s title, date, link, and notice number, and stores them in PostgreSQL. The API exposes paginated access to the collected notices, and a Next.js frontend (with TanStack Query) provides a browsable, paginated list.

How the scraper works

https://iost.tu.edu.np/notices
          │
          ▼
   requests + BeautifulSoup
          │
          ├── title          (h5)
          ├── date           (nep_date)
          ├── link           (a href)
          └── notice_number  (parsed from link)
          │
          ▼
      PostgreSQL

The scraper follows the “next” pagination link on each page and keeps going until there are no more pages. Each notice is deduplicated by notice_number, so re-running the seed only inserts new notices.

API

The GET /notices/ endpoint accepts page and page_size query params and returns:

{
  "total": 42,
  "page": 1,
  "page_size": 10,
  "total_pages": 5,
  "data": [
    { "title": "...", "notice_number": 1234, "link": "...", "date": "..." }
  ]
}

Frontend

A Next.js App Router frontend using TanStack Query to fetch and cache pages. Each page change updates the query key, so the UI refetches automatically without a full reload. Pagination controls move between pages with Previous/Next buttons.

Tech Stack

• Python + BeautifulSoup (scraping)
• FastAPI + SQLModel (API)
• PostgreSQL (persistence)
• Next.js 16 + TanStack Query (frontend)
• Docker / docker-compose (containerized development)

Project Structure

backend/
├── main.py                 # FastAPI app entry point (CORS, startup init)
├── routers.py              # Central router registration
├── database.py             # DB engine & session setup
├── scraper/
│   └── iost-notice-scraper.py
└── notices/
    ├── models/notice.py    # Notice model (unique notice_number)
    ├── routers/            # /notices routes
    ├── schemas/
    ├── services/           # seed_notices logic
    └── __init__.py

Getting Started

Backend

cd backend

uv sync
cp .env.example .env   # set DATABASE_URL

uv run alembic upgrade head
uv run uvicorn main:app --reload

Then seed the database:

curl -X POST http://localhost:8000/notices/seed

Frontend

cd frontend

npm install
cp .env.example .env.local   # set NEXT_PUBLIC_API_URL=http://localhost:8000

npm run dev