Kotlin: High-Performance concurrent file I/O best-practices? - performance

What's the most performant way in Kotlin to allow concurrent file I/O in multi-reader, single-writer fashion?
I have the below, but I'm unsure how much overhead is being created by the coroutine facilities:
AsyncFileChannel, with extension functions to use it in a suspend context
Taken from example here: https://github.com/Kotlin/coroutines-examples/blob/master/examples/io/io.kt
DiskManager class, that uses a custom ReadWriteMutex
Searching for examples of this doesn't turn up much (try searching Github for ReadWriteMutex, there are a tiny handful of Kotlin repos implementing this).
class DiskManagerImpl(file: File) : DiskManager {
private val mutex = ReadWriteMutexImpl()
private val channel = AsynchronousFileChannel.open(file.toPath(),
StandardOpenOption.READ, StandardOpenOption.WRITE)
override suspend fun readPage(pageId: PageId, buffer: MemorySegment) = withContext(Dispatchers.IO) {
mutex.withReadLock {
val offset = pageId * PAGE_SIZE
val bytesRead = channel.readAsync(buffer.asByteBuffer(), offset.toLong())
require(bytesRead == PAGE_SIZE) { "Failed to read page $pageId" }
}
}
override suspend fun writePage(pageId: PageId, buffer: MemorySegment) = withContext(Dispatchers.IO) {
mutex.withWriteLock {
val offset = pageId * PAGE_SIZE
val bytesWritten = channel.writeAsync(buffer.asByteBuffer(), offset.toLong())
require(bytesWritten == PAGE_SIZE) { "Failed to write page $pageId" }
}
}
}
class ReadWriteMutexImpl : ReadWriteMutex {
private val read = Mutex()
private val write = Mutex()
private val readers = atomic(0)
override suspend fun lockRead() {
if (readers.getAndIncrement() == 0) {
read.lock()
}
}
override fun unlockRead() {
if (readers.decrementAndGet() == 0) {
read.unlock()
}
}
override suspend fun lockWrite() {
read.lock()
write.lock()
}
override fun unlockWrite() {
write.unlock()
read.unlock()
}
}
suspend inline fun <T> ReadWriteMutex.withReadLock(block: () -> T): T {
lockRead()
return try {
block()
} finally {
unlockRead()
}
}
suspend inline fun <T> ReadWriteMutex.withWriteLock(block: () -> T): T {
lockWrite()
return try {
block()
} finally {
unlockWrite()
}
}

Related

How to convert ByteReadChannel into Flow<ByteBuffer>

How can I convert io.ktor.utils.io.ByteReadChannel into kotlinx.coroutines.flow.Flow<java.nio.ByteBuffer>?
I use Ktor with this routing:
post("/upload") {
val channel: ByteReadChannel = call.receiveChannel()
val flow: Flow<ByteBuffer> = channel.asByteBufferFlow() // my custom extension method
transaction.execute {
testDao.saveFile(flow)
}
call.respond("OK")
}
The DAO uses R2DBC and Blob like this:
override suspend fun saveFile(input: Flow<ByteBuffer>) {
val connection = requireR2DBCTransactionConnection()
val publisher: Publisher<ByteBuffer> = input.asPublisher()
val statement: Statement = connection.createStatement("insert into bindata (data) values ($1)")
statement.bind(0, Blob.from(publisher))
val count: Int = statement.execute().awaitFirst().rowsUpdated.awaitFirst()
if (count != 1) {
throw IllegalStateException()
}
}
I tried to write this extension method but I failed:
fun ByteReadChannel.asByteBufferFlow(): Flow<ByteBuffer> = object : AbstractFlow<ByteBuffer>() {
override suspend fun collectSafely(collector: FlowCollector<ByteBuffer>) {
/* I have no idea */
}
}
My main problem is that I have not found any similar sample and both ByteBuffer and ByteReadChannel is new for me.

kotlin connect Timeout

I'm trying to do a task with an Asynctask in kotlin. I want to show a dialog message to user In the case that user has a very slow or spotty data connection at the time of use, I'd like to make the AsyncTask timeout after a period of time 5 second .My point is where should l put dialog ? after finally in doInBackground ?
inner class Arr : AsyncTask<String, String, String>() {
val progressDialog = AlertDialog.Builder(this#MainActivity)
val dialogView = layoutInflater.inflate(R.layout.progress_dialog, null)
val message = dialogView.findViewById<TextView>(R.id.message_id)
val dialog = progressDialog.create()
override fun onPreExecute() {
super.onPreExecute()
dialog.setMessage("please wait")
dialog.setCancelable(false)
dialog.show()
}
// for build connection
override fun doInBackground(vararg url: String?): String {
var text: String
val connection = URL(url[0]).openConnection() as HttpURLConnection
connection.connectTimeout = 300
try {
connection.connect()
text = connection.inputStream.use { it.reader().use { reader -> reader.readText() } }
} finally {
dialog.setMessage("Sorry you dont have proper net connectivity..!\nCheck your internet settings or retry.")
dialog.setCancelable(false)
dialog.show()
}
return text
}
override fun onPostExecute(result: String?) {
super.onPostExecute(result)
handleJson(result)
dialog.dismiss();
}
override fun onProgressUpdate(vararg text: String?) {
}
}

LiveData Transformation not getting triggered

I subscribed to ids and search in the ui but i wasn't getting any results so i stepped through with the debugger and found out that the transformation is not getting triggered after the first time. So when i call setIds the first time ids gets updated but for every call after the first one the transformation won't trigger. Same goes for the search.
Any ideas what might possible go wrong?
class MyViewModel : ViewModel() {
private val repository = Repository.sharedInstance
var recentRadius: LiveData<List<RecentRadius>>?
var recentRoute: LiveData<List<RecentRoute>>?
init {
recentRadius = repository.recentRadius()
recentRoute = repository.recentRoute()
}
private val idsInput = MutableLiveData<String>()
fun setIdsInput(textId: String) {
idsInput.value = textId
}
val ids: LiveData<List<String>> = Transformations.switchMap(idsInput) { id ->
repository.ids(id)
}
private val searchInput = MutableLiveData<Search>()
fun setSearchInput(search: Search) {
searchInput.value = search
}
val search: LiveData<SearchResult> = Transformations.switchMap(searchInput) { search ->
when (search.type) {
SearchType.ID -> repository.id(search)
SearchType.RADIUS -> repository.radius(search)
SearchType.ROUTE -> repository.route(search)
}
}
}
The most common reason why transformation don't get triggered is when there is no Observer observing it or the input LiveData is not getting changed.
Below example illustrates use of map when observer is attached in the activity.
Activity
class MainActivity : AppCompatActivity() {
lateinit var mBinding : ActivityMainBinding
private val mViewModel : MainViewModel by lazy {
getViewModel { MainViewModel(this.application) }
}
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
mBinding = DataBindingUtil.setContentView(this, R.layout.activity_main)
mBinding.vm = mViewModel
// adding obeserver
mViewModel.videoName.observe(this, Observer<String> { value ->
value?.let {
//Toast.makeText(this, it, Toast.LENGTH_LONG).show()
}
})
}
}
ViewModel with map
class MainViewModel(val appContext : Application) : AndroidViewModel(appContext) {
private val TAG = "MainViewModel"
var videoData = MutableLiveData<VideoDownload>()
var videoName : LiveData<String>
init {
// Update the data
videoName = Transformations.map(videoData) { "updated : "+it.webUrl }
}
fun onActionClick(v : View) {
// change data
videoData.value = VideoDownload(System.currentTimeMillis().toString())
}
fun onReActionClick(v : View) {
// check data
Toast.makeText(appContext, videoName.value, Toast.LENGTH_LONG).show()
}
}
ViewModel with switchMap
class MainViewModel(val appContext : Application) : AndroidViewModel(appContext) {
private val TAG = "MainViewModel"
var videoData = MutableLiveData<VideoDownload>()
var videoName : LiveData<String>
init {
// Update the data
videoName = Transformations.switchMap(videoData) { modData(it.webUrl) }
}
private fun modData(str: String): LiveData<String> {
val liveData = MutableLiveData<String>()
liveData.value = "switchmap : "+str
return liveData
}
fun onActionClick(v : View) {
// change data
videoData.value = VideoDownload(System.currentTimeMillis().toString())
}
fun onReActionClick(v : View) {
// check data
Toast.makeText(appContext, videoName.value, Toast.LENGTH_LONG).show()
}
}
for me, it was because the observer owner was a fragment. It stopped triggering when navigating to different fragments. I changed the observer owner to the activity and it triggered as expected.
itemsViewModel.items.observe(requireActivity(), Observer {
The view model was defined as a class property:
private val itemsViewModel: ItemsViewModel by lazy {
ViewModelProvider(requireActivity()).get(ItemsViewModel::class.java)
}
If you really want it to be triggered.
fun <X, Y> LiveData<X>.forceMap(
mapFunction: (X) -> Y
): LiveData<Y> {
val result = MutableLiveData<Y>()
this.observeForever {x->
if (x != null) {
result.value = mapFunction.invoke(x)
}
}
return result
}

Android webview image upload

developed a webview app, I have an option to upload image (input type = "file"). In the browser functions normally, but within the webview, it does not. I would like some help to resolve this problem.
because you not post any code, take a look of my code.
It allow webview to upload image from camera or galery
class HandlingWebview(){
private var mFilePathCallback: ValueCallback<Array<Uri>>? = null
private var mCameraPhotoPath: String? = null
companion object {
const val CHOOSE_FILE_REQUEST_CODE = 9685
}
override fun onActivityCreated(savedInstanceState: Bundle?) {
super.onActivityCreated(savedInstanceState)
initObserver()
initView()
}
private val permissionUtils: PermissionUtils by lazy {
PermissionUtils(
this,
trackingService,
getString(R.string.rationale_storage),
Constant.RC_PERMISSIONS_DOWNLOAD_DOCS,
Constant.PERMISSIONS_DOWNLOAD_DOCS
)
}
private fun initView() {
binding.webView.settings.apply {
javaScriptEnabled = true
loadWithOverviewMode = true
useWideViewPort = true
domStorageEnabled = true
}
binding.webView.setOnKeyListener(object : View.OnKeyListener {
override fun onKey(v: View?, keyCode: Int, event: KeyEvent?): Boolean {
if (event?.action == KeyEvent.ACTION_DOWN) {
val webView = v as WebView
when (keyCode) {
KeyEvent.KEYCODE_BACK -> if (webView.canGoBack()) {
webView.goBack()
return true
}
}
}
return false
}
})
binding.webView.apply {
loadUrl(url)
}
binding.webView.webViewClient = object : WebViewClient() {
override fun doUpdateVisitedHistory(view: WebView?, url: String?, isReload: Boolean) {
super.doUpdateVisitedHistory(view, url, isReload)
}
}
binding.webView.webChromeClient = object : WebChromeClient() {
override fun onShowFileChooser(
webView: WebView?,
filePathCallback: ValueCallback<Array<Uri>>?,
fileChooserParams: FileChooserParams?
): Boolean {
if (permissionUtils.isAllPermissionAllowed()) {
// Double check that we don't have any existing callbacks
startActivityChooser(fileChooserParams, filePathCallback)
} else observePermissionResult(permissionUtils.build().asLiveData(), fileChooserParams, filePathCallback)
return true
}
}
}
override fun onRequestPermissionsResult(
requestCode: Int,
permissions: Array<out String>,
grantResults: IntArray
) = permissionUtils.onRequestPermissionsResult(requestCode, permissions, grantResults)
private fun startActivityChooser(
fileChooserParams: WebChromeClient.FileChooserParams?,
filePathCallback: ValueCallback<Array<Uri>>?
) {
mFilePathCallback?.onReceiveValue(null)
mFilePathCallback = filePathCallback
activity?.packageManager?.let {
var takePictureIntent: Intent? = Intent(MediaStore.ACTION_IMAGE_CAPTURE)
if (takePictureIntent?.resolveActivity(it) != null){
var photoFile: File? = null
try {
photoFile = createImageFile()
takePictureIntent.putExtra("PhotoPath", mCameraPhotoPath)
} catch (ex: IOException) {
// Error occurred while creating the File
Timber.i("Unable to create Image File $ex")
}
// Continue only if the File was successfully created
if (photoFile != null) {
mCameraPhotoPath = "file:" + photoFile.absolutePath
takePictureIntent.putExtra(
MediaStore.EXTRA_OUTPUT,
Uri.fromFile(photoFile)
)
} else {
takePictureIntent = null
}
}
val contentSelectionIntent = Intent(Intent.ACTION_GET_CONTENT)
contentSelectionIntent.addCategory(Intent.CATEGORY_OPENABLE)
contentSelectionIntent.type = "image/*"
val intentArray: Array<Intent?> = takePictureIntent?.let { arrayOf(it) } ?: arrayOfNulls(0)
val chooserIntent = Intent(Intent.ACTION_CHOOSER)
chooserIntent.putExtra(Intent.EXTRA_ALLOW_MULTIPLE, true)
chooserIntent.putExtra(Intent.EXTRA_INTENT, contentSelectionIntent)
chooserIntent.putExtra(Intent.EXTRA_TITLE, "Image Chooser")
chooserIntent.putExtra(Intent.EXTRA_INITIAL_INTENTS, intentArray)
startActivityForResult(chooserIntent, CHOOSE_FILE_REQUEST_CODE)
}
}
private fun initObserver() {}
private fun observePermissionResult(
permissionResult: LiveData<Event<PermissionUtils.Companion.PermissionResult>>,
fileChooserParams: WebChromeClient.FileChooserParams?,
filePathCallback: ValueCallback<Array<Uri>>?
) {
permissionResult.observe(viewLifecycleOwner) { event ->
event?.getContentIfNotHandled()?.let {
when (it) {
is PermissionUtils.Companion.PermissionResult.Denied -> {
// pass
}
is PermissionUtils.Companion.PermissionResult.Granted -> {
// pass
}
is PermissionUtils.Companion.PermissionResult.AllGranted -> {
startActivityChooser(fileChooserParams, filePathCallback)
}
}
}
}
}
override fun useCustomBackEvent(): Boolean = true
override fun onBackEvent() {
destroyWebView()
super.onBackEvent()
}
private fun destroyWebView() {
binding.llParent.removeAllViews()
binding.webView.apply {
clearHistory()
clearCache(true)
onPause()
removeAllViews()
destroyDrawingCache()
destroy()
}
}
override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
when (requestCode) {
CHOOSE_FILE_REQUEST_CODE -> {
var results: Array<Uri>? = null
if (resultCode == Activity.RESULT_OK && mFilePathCallback!= null) {
if (data == null) { // take photo from camera
if (mCameraPhotoPath != null) results = arrayOf(Uri.parse(mCameraPhotoPath))
} else { // image picker
data.dataString?.let { results = arrayOf(Uri.parse(it)) }
}
}
mFilePathCallback?.onReceiveValue(results)
mFilePathCallback = null
}
}
super.onActivityResult(requestCode, resultCode, data)
}
#Throws(IOException::class)
private fun createImageFile(): File? {
// Create an image file name
val timeStamp: String = getTodayDateString()
val imageFileName = "JPEG_" + timeStamp + "_"
val storageDir: File = Environment.getExternalStoragePublicDirectory(
Environment.DIRECTORY_PICTURES
)
return File.createTempFile(
imageFileName, /* prefix */
".jpg", /* suffix */
storageDir /* directory */
)
}
}

Cannot call a function from init block because of val property

I'd like to initialize my class's properties.
Because I'm using heavily the functional elements of Kotlin, I'd like to put these initializations to well named functions, to increase readability of my code.
The problem is that I cannot assign a val property, if the code is not in the init block, but in function which is called from the init block.
Is it possible to take apart initialization of a class, to different functions, if the properties are vals?
Here is the code:
val socket: DatagramSocket = DatagramSocket()
val data: ByteArray = "Cassiopeiae server discovery packet".toByteArray()
val broadcastAddresses: List<InetAddress>
init {
socket.broadcast = true
val interfaceAddresses = ArrayList<InterfaceAddress>()
collectValidNetworkInterfaces(interfaceAddresses)
collectBroadcastAddresses(interfaceAddresses)
}
private fun collectValidNetworkInterfaces(interfaceAddresses: ArrayList<InterfaceAddress>) {
NetworkInterface.getNetworkInterfaces().toList()
.filter { validInterface(it) }
.forEach { nInterface -> nInterface.interfaceAddresses.toCollection(interfaceAddresses) }
}
private fun collectBroadcastAddresses(interfaceAddresses: ArrayList<InterfaceAddress>) {
broadcastAddresses = interfaceAddresses
.filter { address -> address.broadcast != null }
.map { it.broadcast }
}
Of course it's not compiling, because collectBroadcastAddresses function tries to reassign the broadcastAddresses val. Although I don't want to put the code of this function to the init block, because it's not obvious what the code is doing, and the function name tells it very nicely.
What can I do in such cases? I'd like to keep my code clean, this is the most important point!
One way of approaching the problem is to use pure functions to initialize fields:
class Operation {
val socket = DatagramSocket().apply { broadcast = true }
val data: ByteArray = "Cassiopeiae server discovery packet".toByteArray()
val broadcastAddresses = collectBroadcastAddresses(collectValidNetworkInterfaces())
private fun collectValidNetworkInterfaces() =
NetworkInterface.getNetworkInterfaces().toList()
.filter { validInterface(it) }
.flatMap { nInterface -> nInterface.interfaceAddresses }
private fun validInterface(it: NetworkInterface?) = true
private fun collectBroadcastAddresses(interfaceAddresses: List<InterfaceAddress>) {
interfaceAddresses
.filter { address -> address.broadcast != null }
.map { it.broadcast }
}
}
Notice how the socket field initialization uses apply extension.
I often find it useful to extract collection manipulation routines into extension methods:
class Operation {
val socket = DatagramSocket().apply { broadcast = true }
val data: ByteArray = "Cassiopeiae server discovery packet".toByteArray()
val broadcastAddresses = NetworkInterface.getNetworkInterfaces()
.collectValidNetworkInterfaces { validInterface(it) }
.collectBroadcastAddresses()
private fun validInterface(it: NetworkInterface?) = true
}
fun Iterable<InterfaceAddress>.collectBroadcastAddresses(): List<InetAddress> =
filter { address -> address.broadcast != null }.map { it.broadcast }
fun Enumeration<NetworkInterface>.collectValidNetworkInterfaces(isValid: (NetworkInterface) -> Boolean = { true }) =
toList()
.filter { isValid(it) }
.flatMap { nInterface -> nInterface.interfaceAddresses }

Resources