move query logic from perl to sql for scalability