{"id":1777,"date":"2022-06-01T07:49:02","date_gmt":"2022-06-01T07:49:02","guid":{"rendered":"http:\/\/dev.technology-architects.com\/bahria_web\/?p=1777"},"modified":"2022-07-26T11:35:36","modified_gmt":"2022-07-26T11:35:36","slug":"content-based-indexing-and-retrieval-of-videos","status":"publish","type":"post","link":"https:\/\/coeai.bahria.edu.pk\/index.php\/content-based-indexing-and-retrieval-of-videos\/","title":{"rendered":"Content-based Indexing and Retrieval of Videos"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t

Content-based Indexing and Retrieval of Videos<\/h2>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

The last decade has witnessed a tremendous increase in the digital multimedia data including images, audio, videos as well as News, entertainment and sports channels. Such enormous collections of videos have opened up a whole new world of challenges to develop smart content retrieval systems allowing users an efficient and effective retrieval of desired content. Traditional video search engines rely on matching the provided query words with user-assigned tags for retrieval purposes. Such systems do not take into accoamunt the actual content of video which itself can serve as a semantic index for robust and efficient retrieval. Content-based search engines, on the other hand, exploit the rich visual, audio and textual content to index and subsequently retrieve videos. Among these, the focus of our current product lies on the textual (News tickers, scorecards, credits etc.) and audio (spoken words) content. More specifically, we have developed a comprehensive video retrieval system that allows users to input (query) keywords and retrieve all videos where the keyword has appeared (either in speech on in the form of caption text). State-of-the-art deep learning-based algorithms are applied for text detection and recognition while an off-the-shelf API is employed for speech recognition. Taking into account the local needs, we have targeted text and audio in two languages, Urdu and English but the system can easily be extended to other languages as well. Once the videos are indexed, users can provide query keyword and all the videos (and frames) where the keyword has appeared are returned to the user. The system is developed as a desktop application while retrieval can also be supported through web if required.<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t

The last decade has witnessed a tremendous increase in the digital multimedia data including images, audio, videos as well as News, <\/span>entertainment<\/span> and sports channels. Such enormous collections of videos have <\/span>opened up<\/span> a whole new world of challenges to develop smart content retrieval systems allowing users an efficient and effective retrieval of desired content. Traditional video search engines rely on matching the provided query words with user-assigned tags for retrieval purposes. Such systems do not <\/span>take into <\/span>account<\/span> the actual content of video which itself can serve as a semantic index for robust and efficient retrieval. Content-based search engines, on the other hand, exploit the rich visual, <\/span>audio<\/span> and textual content to index and subsequently retrieve videos. Among these, the focus of our current product lies on the textual (News tickers, scorecards, credits etc.) and audio (spoken words) content. More specifically, we have developed a comprehensive video retrieval system that allows users to input (query) keywords and retrieve all videos where the keyword has appeared (<\/span>either in<\/span> speech on in the form of caption text). State-of-the-art deep learning-based algorithms are applied for text detection and recognition while an off-the-shelf API is employed for speech recognition. <\/span>Taking into account<\/span> the local needs, we have targeted text and audio in two languages, Urdu and <\/span>English but the system can easily be extended to other languages as well. Once the videos are indexed, users can provide query keyword and all the videos (and frames) where the keyword has appeared are returned to the user. The system is developed as a desktop application while retrieval can also be supported through web if required.<\/span><\/span><\/p>

Funding Body:<\/strong> IGNITE, National Technology Fund.
Partner<\/strong>: Associated Press of Pakistan (APP)<\/p>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t

\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t

Project Team<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/path><\/svg><\/span>\n\t\t\t\t\t\t\t\t<\/path><\/svg><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tKey Researchers<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t
  • Dr. Imran Siddiqi \u2013 Principal Investigator<\/span><\/li>
  • Dr. Shehzad Khalid \u2013 Co-Principal Investigator<\/span><\/li>
  • Dr. Ahmad Salman \u2013 Team Lead (Audio)<\/span><\/li><\/ul><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t
    \n\t\t\t\t\t
    \n\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/path><\/svg><\/span>\n\t\t\t\t\t\t\t\t<\/path><\/svg><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tSoftware Developers<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t
    • Muhammad Atif<\/span><\/li>
    • Osama Zeeshan<\/span><\/li>
    • Muhammad Numan Khan<\/span><\/li>
    • Mazahir Hussain<\/span><\/li>
    • Syed Ghulam Mustafa<\/span><\/li>
    • Wajahat Nawaz<\/span><\/li><\/ul><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t
      \n\t\t\t\t\t
      \n\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/path><\/svg><\/span>\n\t\t\t\t\t\t\t\t<\/path><\/svg><\/span>\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\t\t\t\tResearch Assistants<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t\t
      • Ghulam Ali Mirza \u2013 PhD Scholar<\/span><\/li>
      • Umar Hayat \u2013 MS Student<\/span><\/li>
      • Shanza Ejaz \u2013 MS Student<\/span><\/li>
      • Hamayoun Ahmad \u2013 MS Student<\/span><\/li>
      • Saim Danish \u2013 BS Student<\/span><\/li>
      • Manghangir Sham Sunder \u2013 BS Student<\/span><\/li>
      • Ashbal Sohail \u2013 BS Student<\/span><\/li>
      • Danish Yasin \u2013 BS Student<\/span><\/li><\/ul><\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
        \n\t\t\t\t\t\t
        \n\t\t\t\t\t
        \n\t\t\t
        \n\t\t\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t\t
        \n\t\t\t\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t

        Publications<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
        \n\t\t\t\t
        \n\t\t\t\t\t\t\t
        1. Mirza, A., Zeshan, O., Atif, M., & Siddiqi, I. (2020). Detection and recognition of cursive text from video frames. EURASIP Journal on Image and Video Processing, 2020(1), 1-19.<\/span><\/li>
        2. Mirza, A., & Siddiqi, I. (2020). Recognition of cursive video text using a deep learning framework. IET Image Processing, 14(14), 3444-3455.<\/span><\/li>
        3. Mirza, A., Siddiqi, I., Hayat, U., Atif, M., & Mustufa, S. G. (2020, October). Recognition of Cursive Caption Text Using Deep Learning-A Comparative Study on Recognition Units. In International Conference on Pattern Recognition and Artificial Intelligence (pp. 156-167). Springer, Cham.<\/span><\/li>
        4. Mirza, A., Siddiqi, I., Mustufa, S. G., & Hussain, M. (2019, July). Impact of pre-processing on recognition of cursive video text. In Iberian Conference on Pattern Recognition and Image Analysis (pp. 565-576). Springer, Cham.<\/span><\/li>
        5. Hayat, U., Aatif, M., Zeeshan, O., & Siddiqi, I. (2018, November). Ligature recognition in Urdu caption text using deep convolutional neural networks. In 2018 14th International Conference on Emerging Technologies (ICET) (pp. 1-6). IEEE. (<\/span>Best Paper Award<\/span><\/i><\/b>)<\/span><\/li>
        6. Mirza, A., Fayyaz, M., Seher, Z., & Siddiqi, I. (2018, March). Urdu caption text detection using textural features. In Proceedings of the 2nd Mediterranean Conference on Pattern Recognition and Artificial Intelligence (pp. 70-75). (<\/span>Best Paper Award<\/span><\/i><\/b>).<\/span><\/li><\/ol>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
          \n\t\t\t\t\t\t
          \n\t\t\t\t\t
          \n\t\t\t
          \n\t\t\t\t\t\t
          \n\t\t\t\t
          \n\t\t\t\t\t
          \n\t\t\t\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
          \n\t\t\t\t
          \n\t\t\t

          Downloads<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
          \n\t\t\t\t
          \n\t\t\t\t\t\t\t

          Ground Truth Labeling Tool<\/strong><\/p>

          • Ground truth labeling Software can be downloaded here<\/u> with user manual<\/u>.<\/li>
          • Dataset UTiV can be downloaded here by signing the license agreement.<\/li><\/ul>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
            \n\t\t\t\t\t\t
            \n\t\t\t\t\t
            \n\t\t\t
            \n\t\t\t\t\t\t
            \n\t\t\t\t\t\t
            \n\t\t\t\t\t
            \n\t\t\t
            \n\t\t\t\t\t\t
            \n\t\t\t\t
            \n\t\t\t\t\t
            \n\t\t\t\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
            \n\t\t\t\t
            \n\t\t\t

            Project Demo<\/h3>\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
            \n\t\t\t\t
            \n\t\t\t\t\t
            \n\t\t\t
            <\/div>\t\t\t\t
            \n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t
            \n\t\t\t\t\t\t\t<\/path><\/svg>\t\t\t\t\t\t\tPlay Video<\/span>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
            \n\t\t\t\t
            \n\t\t\t\t\t\t\tOutput of Text Detector<\/strong>\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
            \n\t\t\t\t
            \n\t\t\t\r\n\t\t
            \r\n\t\t\t
            \r\n\t\t\t\t\t\t\t\t\t