title: How to fetch files inside hidden folder using Media Store API on Android 11
body: <p>I need to fetch data inside WhatsApp folders on External Storage.
  As i am targeting API Level 30 i am no longer able to access WhatsApp folders on External Storage. I have implemented <code>Storage Access Framework</code> and got <code>Android/media</code> folder <code>Uri</code> and <code>Document File</code>. And using <code>listFiles()</code> i am able to list files but with <code>filter()</code> and <code>sortedByDescending()</code> functions it becomes very slow.</p>
  <p><strong>What i have tried?</strong></p>
  <ul>
  <li><p>Used Cursor loader with Projection and Selection Arguments but it only
  worked for non hidden folders like <code>WhatsApp Images</code> and <code>WhatsApp Videos</code></p>
  </li>
  <li><p>It returns empty cursor for hidden folder <code>.Statuses</code></p>
  </li>
  <li><p>Tried replacing <code>MediaStore.Video.Media.EXTERNAL_CONTENT_URI</code> with <code>MediaStore.Video.Media.getContentUri(MediaStore.VOLUME_EXTERNAL)</code></p>
  </li>
  </ul>
  <p><strong>What is required?</strong></p>
  <ul>
  <li>List images and videos from .Statuses folder same as i am listing WhatsApp Images using Media Store in HomeActivity.java</li>
  </ul>
  <p><em>Below is my code</em></p>
  <h2>In this activity i get permision to Android/media and set all WhatsApp folders URIs for status fetching and other use, but fetched WhatsApp Images with projection and selection from WhatsApp Images folder</h2>
  <pre><code>class HomeActivity : AppCompatActivity(), InternetListener, PurchasesUpdatedListener,
      CoroutineScope {
      private val exceptionHandler = CoroutineExceptionHandler { context, exception -&gt;
          Toast.makeText(this, exception.message, Toast.LENGTH_LONG).show()
  
      }
      private val dataRepository: DataRepository by inject()
      val tinyDB: TinyDB by inject()
  
      val REQUEST_CODE = 12123
  
      init {
          newNativeAdSetUp = null
      }
  
      val sharedViewModel by viewModel&lt;SharedViewModel&gt;()
  
      val viewModel by viewModel&lt;HomeViewModel&gt;()
  
  
      val handler = CoroutineExceptionHandler { _, exception -&gt;
          Log.d(&quot;CoroutineException&quot;, &quot;$exception handled !&quot;)
      }
      override val coroutineContext: CoroutineContext
          get() = Dispatchers.Main + job + handler
      private lateinit var job: Job
      val sdk30PermissionListener = object : PermissionListener {
          override fun onPermissionGranted() {
              openDocumentTree()
          }
  
          override fun onPermissionDenied(deniedPermissions: MutableList&lt;String&gt;?) {
          }
      }
  
  
      override fun onCreate(savedInstanceState: Bundle?) {
          super.onCreate(savedInstanceState)
          setContentView(R.layout.activity_home)
  
          handlePermissionsByVersion()
      }
  
      private fun handlePermissionsByVersion() {
  
          if (SDK_INT &gt;= Build.VERSION_CODES.R) {
              if ((ContextCompat.checkSelfPermission(
                      this,
                      Manifest.permission.WRITE_EXTERNAL_STORAGE
                  )
                          == PackageManager.PERMISSION_GRANTED) &amp;&amp; (ContextCompat.checkSelfPermission(
                      this,
                      Manifest.permission.READ_EXTERNAL_STORAGE
                  )
                          == PackageManager.PERMISSION_GRANTED)
              ) {
                  //if granted load whatsapp images and some uris setup to viewmodel
                  loadWhatsAppImages()
                  if (arePermissionsGranted()) {
                      if (dataRepository.mrWhatsAppImages == null || dataRepository.mrWhatsAppBusinessImages == null) {
                          setUpWAURIs()
                      }
                  }
              } else {
                  TedPermission.with(this)
                      .setPermissionListener(sdk30PermissionListener)
                      .setDeniedMessage(&quot;If you reject permission,you can not use this service\n\nPlease turn on permissions at [Setting] &gt; [Permission]&quot;)
                      .setPermissions(
                          Manifest.permission.WRITE_EXTERNAL_STORAGE,
                          Manifest.permission.READ_EXTERNAL_STORAGE
                      )
                      .check()
              }
  
          }
      }
  
      override fun onActivityResult(requestCode: Int, resultCode: Int, @Nullable data: Intent?) {
          super.onActivityResult(requestCode, resultCode, data)
  
  
          if (resultCode == RESULT_OK &amp;&amp; requestCode == REQUEST_CODE) {
              if (data != null) {
                  //this is the uri user has provided us
                  val treeUri: Uri? = data.data
                  if (treeUri != null) {
                      sharedViewModel.treeUri = treeUri
                      val decoded = Uri.decode(treeUri.toString())
                      Log.i(LOGTAG, &quot;got uri: ${treeUri.toString()}&quot;)
                      // here we should do some checks on the uri, we do not want root uri
                      // because it will not work on Android 11, or perhaps we have some specific
                      // folder name that we want, etc
                      if (Uri.decode(treeUri.toString()).endsWith(&quot;:&quot;)) {
                          showWrongFolderSelection()
                          return
                      }
                      if (!decoded.equals(Constants.WHATSAPP_MEDIA_URI_DECODED)) {
                          showWrongFolderSelection()
                          return
                      }
                      // here we ask the content resolver to persist the permission for us
                      val takeFlags: Int = Intent.FLAG_GRANT_READ_URI_PERMISSION or
                              Intent.FLAG_GRANT_WRITE_URI_PERMISSION
                      contentResolver.takePersistableUriPermission(
                          treeUri,
                          takeFlags
                      )
                      val treeUriAsString = treeUri.toString()
                      tinyDB.putString(&quot;FOLDER_URI&quot;, treeUriAsString)
                      if (SDK_INT &gt;= Build.VERSION_CODES.R) {
                          setupPaths()
                      }
  
                  }
              }
          }
      }
  
      private fun setupPaths() {
          setUpOverlay()
          fetchWhatsAppRootURIs(
              this,
              sharedViewModel,
              dataRepository,
              tinyDB
          ) {
              fetchWhatsAppBusinessRootURIs(
                  this,
                  sharedViewModel,
                  dataRepository,
                  tinyDB
              ) {
                  tinyDB.putBoolean(&quot;WARootPathsDone&quot;, true)
                  removeOverlay()
              }
          }
  
  
      }
  
      override fun onDestroy() {
          dialogHandler.removeCallbacksAndMessages(null)
          super.onDestroy()
      }
      val loadmanagerImages = object : LoaderManager.LoaderCallbacks&lt;Cursor&gt; {
          val whatsAppImagesArrayList = arrayListOf&lt;File&gt;()
  
  
          override fun onCreateLoader(id: Int, args: Bundle?): Loader&lt;Cursor&gt; {
              var location: File = File(
                  Environment.getExternalStorageDirectory()
                      .toString() + Constants.whatsapp_images_path
              )
              if (!location.exists()) {
                  location = File(
                      Environment.getExternalStorageDirectory()
                          .toString() + Constants.whatsapp_images_path11
                  )
              }
  
              if (location != null &amp;&amp; location.exists()) {
                  whatsAppImagesArrayList.clear()
                  Timber.e(&quot;checkLoaded-onCreateLoader $id&quot;)
                  if (id == 0) {
                      var folder = location.absolutePath
                      val projection = arrayOf(
                          MediaStore.MediaColumns.DATA,
                          MediaStore.MediaColumns.DATE_MODIFIED
                      )
                      val selection = MediaStore.Images.Media.DATA + &quot; like ? &quot;
                      val selectionArgs: String = &quot;%$folder%&quot;
  
                      return CursorLoader(
                          this@HomeActivity,
                          MediaStore.Images.Media.EXTERNAL_CONTENT_URI,
                          projection,
                          selection,
                          arrayOf(selectionArgs),
                          &quot;${MediaStore.Images.Media.DATE_MODIFIED} DESC&quot;
                      )
                  }
              }
  
              return null!!
          }
  
          override fun onLoadFinished(loader: Loader&lt;Cursor&gt;, cursor: Cursor?) {
              Timber.e(&quot;checkLoaded-onLoadFinished&quot;)
              var absolutePathOfImage: String
              if (loader.id == 0) {
                  cursor?.let {
                      val columnIndexData = cursor.getColumnIndexOrThrow(MediaStore.MediaColumns.DATA)
                      GlobalScope.launch(Dispatchers.Main + exceptionHandler) {
  
                          async(Dispatchers.IO + exceptionHandler) {
                              while (!cursor.isClosed &amp;&amp; cursor.moveToNext() == true) {
                                  absolutePathOfImage = cursor.getString(columnIndexData!!)
                                  whatsAppImagesArrayList.add(File(absolutePathOfImage))
  
                              }
                          }.await()
                          LoaderManager.getInstance(this@HomeActivity).destroyLoader(0)
                          Timber.e(&quot;checkLoaded-Completion&quot;)
                          galleryViewModel.whatsAppImagesList.postValue(whatsAppImagesArrayList)
                      }
  
  
                  }
              }
          }
  
          override fun onLoaderReset(loader: Loader&lt;Cursor&gt;) {
          }
  
      }
  
      fun loadWhatsAppImages() {
          try {
              tinyDB.putBoolean(&quot;whatsAppMediaLoadCalled&quot;, true)
              LoaderManager.getInstance(this).initLoader(
                  0,
                  null,
                  loadmanagerImages
              )
          } catch (e: RuntimeException) {
              Log.e(&quot;exVideos &quot;, &quot;ex : ${e.localizedMessage}&quot;)
          }
  
      }
  
  
      companion object {
          const val ANDROID_DOCID = &quot;primary:Android/media/&quot;
          const val EXTERNAL_STORAGE_PROVIDER_AUTHORITY = &quot;com.android.externalstorage.documents&quot;
          private val androidUri = DocumentsContract.buildDocumentUri(
              EXTERNAL_STORAGE_PROVIDER_AUTHORITY, ANDROID_DOCID
          )
          val androidTreeUri = DocumentsContract.buildTreeDocumentUri(
              EXTERNAL_STORAGE_PROVIDER_AUTHORITY, ANDROID_DOCID
          )
      }
  
      private fun openDocumentTree() {
          val uriString = tinyDB.getString(&quot;FOLDER_URI&quot;, &quot;&quot;)
          when {
              uriString == &quot;&quot; -&gt; {
                  Log.w(LOGTAG, &quot;uri not stored&quot;)
                  askPermission()
              }
              arePermissionsGranted() -&gt; {
              }
              else -&gt; {
                  Log.w(LOGTAG, &quot;uri permission not stored&quot;)
                  askPermission()
              }
          }
      }
      // this will present the user with folder browser to select a folder for our data
      private fun askPermission() {
          val intent = Intent(Intent.ACTION_OPEN_DOCUMENT_TREE)
          intent.putExtra(DocumentsContract.EXTRA_INITIAL_URI, androidUri)
          startActivityForResult(intent, REQUEST_CODE)
      }
  
      private fun arePermissionsGranted(): Boolean {
          var uriString = tinyDB.getString(&quot;FOLDER_URI&quot;, &quot;&quot;)
          val list = contentResolver.persistedUriPermissions
          for (i in list.indices) {
              val persistedUriString = list[i].uri.toString()
              if (persistedUriString == uriString &amp;&amp; list[i].isWritePermission &amp;&amp; list[i].isReadPermission) {
                  return true
              }
          }
          return false
      }
  
      private fun showWrongFolderSelection() {
          val layoutInflaterAndroid = LayoutInflater.from(this)
          val mView = layoutInflaterAndroid.inflate(R.layout.layout_dialog_wrong_folder, null)
          val builder = AlertDialog.Builder(this, R.style.ThemePageSearchDialog)
          builder.setView(mView)
          val alertDialog = builder.show()
          alertDialog.setCancelable(false)
          val btnOk = mView.findViewById(R.id.tvExit) as TextView
          val tvCancel = mView.findViewById(R.id.tvCancel) as TextView
          btnOk.setOnClickListener {
              alertDialog.dismiss()
              openDocumentTree()
          }
          tvCancel.setOnClickListener {
              alertDialog.dismiss()
          }
  
      }
  
      private fun setUpWAURIs() {
          dataRepository.mrWhatsAppImages =
              getDocumentFileFromStringURI(
                  this,
                  tinyDB.getString(&quot;mrWhatsAppImages&quot;)
              )
          dataRepository.mrWhatsAppVN =
              getDocumentFileFromStringURI(
                  this,
                  tinyDB.getString(&quot;mrWhatsAppVN&quot;)
              )
          dataRepository.mrWhatsAppDocs =
              getDocumentFileFromStringURI(
                  this,
                  tinyDB.getString(&quot;mrWhatsAppDocs&quot;)
              )
          dataRepository.mrWhatsAppVideo =
              getDocumentFileFromStringURI(
                  this,
                  tinyDB.getString(&quot;mrWhatsAppVideo&quot;)
              )
          dataRepository.mrWhatsAppAudio =
              getDocumentFileFromStringURI(
                  this,
                  tinyDB.getString(&quot;mrWhatsAppAudio&quot;)
              )
          dataRepository.WhatsAppStatuses =
              getDocumentFileFromStringURIStatuses(
                  this,
                  tinyDB.getString(&quot;WhatsAppStatuses&quot;)
              )
  
  
  
          dataRepository.mrWhatsAppBusinessImages =
              getDocumentFileFromStringURI(
                  this,
                  tinyDB.getString(&quot;mrWhatsAppBusinessImages&quot;)
              )
          dataRepository.mrWhatsAppBusinessVN =
              getDocumentFileFromStringURI(
                  this,
                  tinyDB.getString(&quot;mrWhatsAppBusinessVN&quot;)
              )
          dataRepository.mrWhatsAppBusinessDocs =
              getDocumentFileFromStringURI(
                  this,
                  tinyDB.getString(&quot;mrWhatsAppBusinessDocs&quot;)
              )
          dataRepository.mrWhatsAppBusinessVideo =
              getDocumentFileFromStringURI(
                  this,
                  tinyDB.getString(&quot;mrWhatsAppBusinessVideo&quot;)
              )
          dataRepository.mrWhatsAppBusinessAudio =
              getDocumentFileFromStringURI(
                  this,
                  tinyDB.getString(&quot;mrWhatsAppBusinessAudio&quot;)
              )
          dataRepository.WhatsAppBusinessStatuses =
              getDocumentFileFromStringURIStatuses(
                  this,
                  tinyDB.getString(&quot;WhatsAppBusinessStatuses&quot;)
              )
      }
  
      fun setUpOverlay() {
          val dialogfragment = FullScreenLoadingDialog()
          dialogfragment.isCancelable = false
          dialogfragment.setisAdmobAd(true)
          val ft: FragmentTransaction =
              supportFragmentManager.beginTransaction()
          ft.add(dialogfragment, &quot;DialogFragment_FLAG&quot;)
          ft.commitAllowingStateLoss()
      }
  
      fun removeOverlay() {
          val fragment: Fragment? = supportFragmentManager.findFragmentByTag(&quot;DialogFragment_FLAG&quot;)
          if (fragment != null &amp;&amp; fragment is DialogFragment) {
              fragment.dismissAllowingStateLoss()
          }
      }
      fun fetchWhatsAppRootURIs(
          context: Context,
          sharedViewModel: SharedViewModel,
          dataRepository: DataRepository,
          tinyDB: TinyDB, completed: () -&gt; Unit
  
      ) {
          val selectedPackageName = Constants.WHATSAPP_PKG_NAME
          val selectedRootName = Constants.WHATSAPP_ROOT_NAME
          var waImages: DocumentFile? = null
          var waVN: DocumentFile? = null
          var waDocs: DocumentFile? = null
          var waVideos: DocumentFile? = null
          var waAudio: DocumentFile? = null
          var waStatus: DocumentFile? = null
          if (Build.VERSION.SDK_INT &gt;= Build.VERSION_CODES.R &amp;&amp; sharedViewModel.treeUri != null) {
              CoroutineScope(Dispatchers.Main).launch {
                  async(Dispatchers.IO) {
                      val dir = DocumentFile.fromTreeUri(
                          context,
                          sharedViewModel.treeUri!!
                      )
                      dir?.listFiles()?.forEach {
                          if (it.name.equals(selectedPackageName)) {
                              it.listFiles().forEach {
                                  if (it.name.equals(selectedRootName)) {
                                      it.listFiles().forEach {
                                          if (it.name.equals(Constants.WHATSAPP_MEDIA_FOLDER_NAME)) {
                                              it.listFiles().forEach {
                                                  if (it.name.equals(Constants.FOLDER_NAME_WHATSAPP_IMAGES)) {
                                                      waImages = it
  
                                                  } else if (it.name.equals(Constants.FOLDER_NAME_WHATSAPP_VN)) {
                                                      waVN = it
  
                                                  } else if (it.name.equals(Constants.FOLDER_NAME_WHATSAPP_DOCUMENTS)) {
                                                      waDocs = it
                                                  } else if (it.name.equals(Constants.FOLDER_NAME_WHATSAPP_VIDEO)) {
                                                      waVideos = it
  
                                                  } else if (it.name.equals(Constants.FOLDER_NAME_WHATSAPP_AUDIO)) {
                                                      waAudio = it
  
                                                  } else if (it.name.equals(Constants.FOLDER_NAME_STATUSES)) {
                                                      waStatus = it
  
                                                  }
  
                                              }
  
  
                                          }
                                      }
                                  }
                              }
                          }
  
  
                      }
                  }.await()
                  Timber.e(&quot;processStatusFetch:Done&quot;)
                  tinyDB.putString(&quot;mrWhatsAppImages&quot;, waImages?.uri.toString())
                  tinyDB.putString(&quot;mrWhatsAppVN&quot;, waImages?.uri.toString())
                  tinyDB.putString(&quot;mrWhatsAppDocs&quot;, waImages?.uri.toString())
                  tinyDB.putString(&quot;mrWhatsAppVideo&quot;, waImages?.uri.toString())
                  tinyDB.putString(&quot;mrWhatsAppAudio&quot;, waImages?.uri.toString())
                  tinyDB.putString(&quot;WhatsAppStatuses&quot;, waStatus?.uri.toString())
  
                  dataRepository.mrWhatsAppImages = waImages
                  dataRepository.mrWhatsAppVN = waVN
                  dataRepository.mrWhatsAppDocs = waDocs
                  dataRepository.mrWhatsAppVideo = waVideos
                  dataRepository.mrWhatsAppAudio = waAudio
                  dataRepository.WhatsAppStatuses = waStatus
                  completed()
  
  
              }
          }
      }
  </code></pre>
  <h2>Here i am using .Statuses folder URI to list DocumentFiles and display but this way it is slow</h2>
  <pre><code>class StatusImageFragment : Fragment(), StatusListener, CoroutineScope {
  
      companion object {
          fun newInstance() = StatusImageFragment()
      }
  
      val handler = CoroutineExceptionHandler { _, exception -&gt;
          Log.d(&quot;CoroutineException&quot;, &quot;$exception handled !&quot;)
      }
      override val coroutineContext: CoroutineContext
          get() = Dispatchers.Main + job + handler
      private lateinit var job: Job
  
      private var adapterSDK30 = StatusImageAdapterSDK30()
      private var no_image: ImageView? = null
      private var no_image_txt: TextView? = null
  
      val tinyDB: TinyDB by inject()
      val sharedViewModel by viewModel&lt;SharedViewModel&gt;()
      private val dataRepository: DataRepository by inject()
  
      override fun onCreateView(
          inflater: LayoutInflater, container: ViewGroup?,
          savedInstanceState: Bundle?
      ): View? {
          job = Job()
          return inflater.inflate(R.layout.status_image_fragment, container, false)
      }
  
      override fun onViewCreated(view: View, savedInstanceState: Bundle?) {
          super.onViewCreated(view, savedInstanceState)
  
          swipeRefresh(false, false)
      }
  
  
      public fun swipeRefresh(isReloadRequired: Boolean, isFromModeChanged: Boolean) {
          try {
              if (Build.VERSION.SDK_INT &gt;= Build.VERSION_CODES.R) {
                  if (isFromModeChanged) {
                      status_image_recycler.visibility = View.GONE
                      progressbar.visibility = View.VISIBLE
                      no_image?.let {
                          it.visibility = View.GONE
                      }
                      no_image_txt?.let {
                          it.visibility = View.GONE
                      }
                      go_to_app?.let {
                          it.visibility = View.GONE
                      }
                  } else {
                      if (adapterSDK30.listImages == null || adapterSDK30.listImages.size == 0) {
                          no_image?.let {
                              it.visibility = View.GONE
                          }
                          no_image_txt?.let {
                              it.visibility = View.GONE
                          }
                          go_to_app?.let {
                              it.visibility = View.GONE
                          }
                          progressbar.visibility = View.VISIBLE
                      }
                  }
                  if (isReloadRequired) {
                      processStatusFetchFromChild({
                          sharedViewModel.statusImages.observe(viewLifecycleOwner, Observer {
                              val arrayList = it
                              adapterSDK30.listImages = arrayList
                              postFetchingExecutionSDK30()
                          })
                      })
  
                  } else {
                      sharedViewModel.statusImages.observe(viewLifecycleOwner, Observer {
                          val arrayList = it
                          adapterSDK30.listImages = arrayList
                          adapterSDK30.listImages = it
                          postFetchingExecutionSDK30()
                      })
                  }
  
              }
          } catch (ex: Exception) {
              ex.printStackTrace()
  
          }
  
      }
  
      private fun postFetchingExecutionSDK30() {
          progressbar.visibility = View.GONE
          status_image_recycler.visibility = View.VISIBLE
          if (adapterSDK30!!.listImages != null &amp;&amp; adapterSDK30!!.listImages.size &gt; 0) {
              no_image?.let {
                  it.visibility = View.GONE
              }
              no_image_txt?.let {
                  it.visibility = View.GONE
              }
              go_to_app?.let {
                  it.visibility = View.GONE
              }
          } else {
              no_image?.let {
                  it.visibility = View.VISIBLE
              }
              no_image_txt?.let {
                  it.visibility = View.VISIBLE
              }
              go_to_app?.let {
                  it.visibility = View.VISIBLE
              }
          }
          adapterSDK30!!.notifyDataSetChanged()
  
          status_img_swipe.isRefreshing = false
      }
  
  
      override fun onDestroyView() {
          job.cancel()
          super.onDestroyView()
      }
  
  
      fun processStatusFetchFromChild(completed: () -&gt; Unit) {
          val statusSelection = tinyDB.getInt(Constants.status_accounts)
          if (statusSelection == 0 || statusSelection == 1) {
              if (dataRepository.WhatsAppStatuses == null) {
                  (activity as StatusActivity).setUpWAURIs()
              }
              var documentFileStatuses: DocumentFile? = dataRepository.WhatsAppStatuses
              if (statusSelection == 1) {
                  documentFileStatuses = dataRepository.WhatsAppBusinessStatuses
              }
              if (documentFileStatuses != null) {
                  launch(Dispatchers.Main) {
                      val statusImages1 = arrayListOf&lt;DocumentFile&gt;()
  
                      async(Dispatchers.IO) {
                          //this takes time ; want to fetch this same as WhatsApp Gallery
                          statusImages1.addAll(documentFileStatuses!!.listFiles().filter {
                              it.mimeType.equals(Constants.MIME_TYPE_IMG_PNG) || it.mimeType.equals(
                                  Constants.MIME_TYPE_IMG_JPG
                              ) || it.mimeType.equals(Constants.MIME_TYPE_IMG_JPEG)
                          }.sortedByDescending { it.lastModified() })
                      }.await()
                      Timber.e(&quot;processStatusFetch:Done&quot;)
                      sharedViewModel.statusImages.postValue(statusImages1)
                      completed()
                  }
              } else {
                  Timber.e(&quot;processStatusFetch:Done&quot;)
                  sharedViewModel.statusImages.postValue(arrayListOf&lt;DocumentFile&gt;())
                  completed()
              }
          } else {
              Timber.e(&quot;processStatusFetch:Done&quot;)
              sharedViewModel.statusImages.postValue(arrayListOf&lt;DocumentFile&gt;())
              completed()
  
  
          }
      }
  
  }
  </code></pre>
  <p>Please note WhatsApp folder path which i used is</p>
  <pre><code>val whatsapp_images_path11 = &quot;/Android/media/“ +&quot;com.whatsapp&quot; +&quot;/WhatsApp/Media/WhatsAppImages/&quot;
  </code></pre>
  <p><strong>How i can use MediaStore in this case so that i don't need to use sort and filter functions of list? Its not important to get java.io File only i can work with URIs as well.</strong></p>