Jetpack Compose: Firing onClick events using a remote control in Android TV - android-jetpack-compose

I'm building my Android TV app using Jetpack Compose, and I'm trying to fire some onClick events on some Text components.
I've implemented the Modifier.focusable, so it can be focused using the remote control, and I've implemented Modifier.clickable to be launched when, well, the component is clicked.
However, when I launch the app on an emulator, I can focus and select the component properly, as I can see the change on the background color, but I can't fire the event inside Modifier.clickable when pressing on the OK button on my remote control (in my case it's KEYCODE_DPAD_CENTER). The event is fired if I click with the mouse inside the emulator, however.
Here is my code
#Composable
fun FocusablePill(text: String, focusRequester: FocusRequester = FocusRequester()) {
val interactionSource = remember { MutableInteractionSource() }
val isFocused by interactionSource.collectIsFocusedAsState()
val isPressed by interactionSource.collectIsPressedAsState()
val color = if (isFocused || isPressed) action else lightTranslucent_10
val shape = RoundedCornerShape(CornerSize(24.dp))
Text(
text = text,
color = MaterialTheme.colors.onPrimary,
style = MaterialTheme.typography.button,
modifier = Modifier
.focusRequester(focusRequester)
.focusable(
interactionSource = interactionSource
)
.clickable(
interactionSource = interactionSource,
indication = null //this is just cosmetic, setting LocalIndication.current still doesn't work
) {
onCommandEntered(text)
}
.background(color, shape)
.padding(16.dp, 8.dp)
)
}
I've also tried with Modifier.selectable, but the result is the same. Event is only fired on mouse click. Also, using Button components doesn't work either.

For future reference, this was fixed and should be working as of 1.1.0-beta01. Both the Dpad center and the enter key will trigger a click on the focused view now. If you want to handle other keys (e.g., a game controller), you can use Modifier.onKeyEvent.

Related

What is the difference between the different options under PointerIconDefaults in jetpack compose

Column(
Modifier.padding(top = 300.dp).pointerHoverIcon(PointerIconDefaults.Text)) {
SelectionContainer {
Column {
Text("Selectable text")
Text(
modifier = Modifier.pointerHoverIcon(PointerIconDefaults.Hand, true),
text = "Selectable text with hand"
)
}
}
Text("Just text with global pointerIcon")
}
PointerIconDefaults.Text:
PointerIconDefaults.Hand:
I don't see any difference between PointerIconDefaults.Hand and PointerIconDefaults.Text
Not sure what you expect from using this modifier since your image shows a floating toolbar. If you need to change how floating toolbar looks like, check out this answer. If you need to change the selection color, check out this one.
Modifier.pointerHoverIcon change the appearance of the mouse pointer over the item. To see this, for example you can connect a BLE mouse to your Android device, not sure if this can be checked in the Emulator.
Any Text has PointerIconDefaults.Text by default.

How to make ClickableText accessible to screen readers

This code creates a ClickableText element in Jetpack Compose Composable:
ClickableText(
text = forgotPasswordAnnotatedString,
onClick = {
context.startActivity(intent)
},
modifier = Modifier
.padding(top = mediumPadding)
)
The annotated string is defined here to make the text look like a link:
val forgotPasswordAnnotatedString = buildAnnotatedString {
append(stringResource(R.string.forgot_password))
addStyle(
style = SpanStyle(
textDecoration = TextDecoration.Underline,
color = Color.White,
fontSize = 16.sp,
fontWeight = FontWeight.Medium
),
start = 0,
end = 21,
)
}
When I encounter this text using the TalkBalk screen reader in Android, the screenreader does not make it clear that this is clickable text that will do something which tapped on. The reader just reads the text.
Is there a way to make it clear to the screen reader that this text is interactive? Otherwise should I just use a button and style it to look like a link?
It looks like your intention is for the whole text to be clickable? In which you best option is probably a TextButton as suggested by
Gabriele Mariotti.
But if you wan't only part of the link to be clickable, or to have multiple clickable sections, the best I've been able to land on is to draw an invisible box overtop of the Text. It means that I can control the touch target of the clickable area to be at least 48.dp and can use the semantics{} modifier to control how a screen reader interprets it.
Would welcome any suggestions.
// remember variables to hold the start and end position of the clickable text
val startX = remember { mutableStateOf(0f) }
val endX = remember { mutableStateOf(0f) }
// convert to Dp and work out width of button
val buttonPaddingX = with(LocalDensity.current) { startX.value.toDp() }
val buttonWidth = with(LocalDensity.current) { (endX.value - startX.value).toDp() }
Text(
text = forgotPasswordAnnotatedString,
onTextLayout = {
startX.value = it.getBoundingBox(0).left // where 0 is the start index of the range you want to be clickable
endX.value = it.getBoundingBox(21 - 1).right // where 21 is the end index of the range you want to be clickable
}
)
Note that buttonPaddingX is relative to the Text position not the Window, so you may have to surround both in a Box{} or use ConstraintLayout.
Then to draw the invisible box
Box(modifier = Modifier
.sizeIn(minWidth = 48.dp, minHeight = 48.dp) // minimum touch target size
.padding(start = buttonPaddingX)
.width(buttonWidth)
// .background(Color.Magenta.copy(alpha = 0.5f)) // uncomment this to debug where the box is drawn
.clickable(onClick = { context.startActivity(intent) })
.semantics {
// tell TalkBack whatever you need to here
role = Role.Button
contentDescription = "Insert button description here"
}
)
In my code I'm using pushStringAnnotation(TAG, annotation) rather than reference string indexes directly. That way I can get the start and end index of the clickable area with annotatedString.getStringAnnotations(TAG,0,annotatedString.length).first(). Useful if there a multiple links within the text.
It's disappointing that ClickableText doesn't have accessibility in mind from the get go, hopefully we'll be able to use it again in a future update.
Adding .semantics.contentDescription to the Modifier changes what is read by the screen reader. I had to word contentDescription to make it clear that this was a link to reset the your password.
The screen reader still doesn't recognize the element a clickable but hopefully the description will be useful to convey to the user that this element is interactive.
ClickableText(
text = forgotPasswordAnnotatedString,
onClick = {
context.startActivity(intent)
},
modifier = Modifier
.padding(top = mediumPadding)
// new code here:
.semantics {
contentDescription = "Forgot your password? link"
}
)

Vaadin 8 using an image as a button. Doesn't fire click event

I'm building a Vaadin 8 app ( first one for me ). I am using the designer to generate the UI. I've added several buttons to the dashboard which should fire a function when clicked. For some reason nothing fires when the image is clicked. Below is all the code that is involved. Can anyone see what I'm doing wrong?
This is the code from the .html file:
<vaadin-horizontal-layout responsive width-full margin>
**<vaadin-image icon="theme://images/properties.png" style-name="my-image-button" responsive alt="" _id="imagePropertyInfo"></vaadin-image>**
<vaadin-image icon="theme://images/occupants.png" responsive alt="" _id="imageOccupants"></vaadin-image>
<vaadin-image icon="theme://images/vendors.png" responsive alt="" _id="imageVendors"></vaadin-image>
</vaadin-horizontal-layout>
Here is the scss
.my-image-button
{
cursor: pointer;
}
Here is the code from the Dashboard UI
public DashboardHomeView( OnCallUI onCallUI )
{
this.onCallUI = onCallUI;
// Make it disabled until a property is selected
**imagePropertyInfo.setEnabled( false );
imagePropertyInfo.setStyleName( "my-image-button" );**
fetchPropertyBasicInfo();
}
protected void fetchPropertyBasicInfo()
{
List<PropertyProfileBasic> listOfPropertyProfiles = new ArrayList<PropertyProfileBasic>( OnCallUI.myStarService.fetchAllPropertyProfileBasicInformation() );
comboBoxGeneric.setCaption( "Select a Property" );
comboBoxGeneric.setItemCaptionGenerator( aProperty -> aProperty.toString() );
comboBoxGeneric.setItems( listOfPropertyProfiles );
comboBoxGeneric.addValueChangeListener( event -> fetchOccupantBasicInfo( event ) );
comboBoxGeneric.focus();
}
protected void fetchOccupantBasicInfo( ValueChangeEvent<PropertyProfileBasic> event )
{
// Fetch all the occupants for the selected property
if( event.getValue().getPropertyNo() != null )
{
// Fetch a list of occupant basic info for the selected property
List<OccupantProfileBasic> listOfOccupantProfiles = new ArrayList<OccupantProfileBasic>( OnCallUI.myStarService.fetchOccupantProfileBasicByPropertyNo( event.getValue().getPropertyNo() ) );
// Clear the existing grid et al
gridContainer.removeAllComponents();
// Add the occupant grid
occupantGrid = new OccupantProfileBasicGrid( listOfOccupantProfiles );
// Show the grid
gridContainer.addComponents( new Label( "Occupants" ), occupantGrid );
// Set the dashboard buttons to enabled now a property is selected
**imagePropertyInfo.setEnabled( true );
// Add the property info button
imagePropertyInfo.addClickListener( e -> fetchPropertyInformation() );**
}
}
protected void fetchPropertyInformation()
{
Notification.show( "Yo!", "You clicked!", Notification.Type.HUMANIZED_MESSAGE );
}
I assume you are using GridLayout. I am recommending another approach. Use Button, and set the button style to be borderless (apparently you want something like that. The icon of the button can be image from your theme, using ThemeResource. "Pseudo code" is something like this:
ThemeResource icon = new ThemeResource("/images/properties.png");
Button imagePropertyInfo = new Button(icon);
imagePropertyInfo.setStyleName(ValoTheme.BUTTON_BORDERLESS);
imagePropertyInfo.addClickListener( e -> fetchPropertyInformation() );
Note also, JavaDoc of Image component says.
"public Registration addClickListener(MouseEvents.ClickListener listener)
Add a click listener to the component. The listener is called whenever the user clicks inside the component. Depending on the content the event may be blocked and in that case no event is fired."
I think it does not like your way of setting image with theme, without using Resource.
If you want to remove the focus highlight of the button, it should be possible via this CSS rule:
.v-button-link:after {
content: none;
}
Also it is worth of mentioning that Image is not Focusable, while Button is. This means that even that Image can have click listener, it is not reached by keyboard navigation, Button is Focusable and is reached by tabbing etc. So using Button instead of Image makes your application more accessible.

addEventListener on youtube API with greasemonkey

I would like to use Greasemonkey to access some API objects of youtube videos while I'm in fullscreen mode.
It could be useful to have mouse clicks and position relative to screen.
This, to detect fullscreen mode, doesn't work:
window.fullScreen
I tried also to add mouse event detection to yt player, with this:
var player = document.getElementById('movie_player');
player.addEventListener("click", interceptClicks,true);
but it doesn't fire that func.
I tried also to inject some code like this:
function GM_main () {
var playerNode = document.getElementById ("movie_player");
playerNode.addEventListener ('click', interceptClicks ,false);
}
addJS_Node (null, null, GM_main);
function addJS_Node (text, s_URL, funcToRun, runOnLoad) {
var D = document;
var scriptNode = D.createElement ('script');
if (runOnLoad) {
scriptNode.addEventListener ("load", runOnLoad, false);
}
scriptNode.type = "text/javascript";
if (text) scriptNode.textContent = text;
if (s_URL) scriptNode.src = s_URL;
if (funcToRun) scriptNode.textContent = '(' + funcToRun.toString() + ')()';
var targ = D.getElementsByTagName ('head')[0] || D.body || D.documentElement;
targ.appendChild (scriptNode);
}
I tried also to make a:
window.addEventListener('click', interceptClicks, false);
This works, BUT only in all areas different from the youtube flash player in non-fullscreen mode and in fullscreen mode, obviously none area, as there is only the player visible...
EDIT:
I made a partial progress indeed.
I created a button element with
btn.addEventListener("click", function () { player.mozRequestFullScreen();}, false)
This way flash video enters in Firefox fullscreen mode and so it receives the wheel events fired by the
window.addEventlistener('DOMMouseScroll', .....etc)
Besides, the fullscreen mode is detected by
window.fullScreen
Also, all keys (event) are detectable, but ESC; not again the mouse clicks..
There is a drawback:
Once in fullscreen, SOMETIMES if you click the left mouse button it suddenly exits fullscreen mode... SOMETIMES instead it stays normally full...
To exit it's not sufficient to press ESC, you need to press the normal flash fullscreen button on the lower right + ESC.
Some rare times it blocks itself in Fullscreen mode and you can't exit. You should press ctrl+alt+canc and then it appears firefox "block script" dialog box.
Why that odd behaviour and how to prevent it?
Ideally the best should be: intercept mouse click on the lower right flash fullscreen button, block it, redirect the call to mozFullscreen and block the fullscreen mode until you press ESC.
But I dont' think it's possible, is it?

getting mouse click screenX and screenY in as3

I want to get mouse click screen coordinates (by clicking outside AIR application window)
I tried the following, but i don't get anything, it seem like the ScreenMouseEvent.CLICK event not dispatched.
public function Main():void
{
if (NativeApplication.supportsSystemTrayIcon)//testExpression return true
{
SystemTrayIcon(NativeApplication.nativeApplication.icon).
addEventListener(ScreenMouseEvent.CLICK, click);
}
}
private function click(e:ScreenMouseEvent):void
{
trace(e.screenX);//nothing displayed :(
}
The ScreenMouseEvent is dispatched by the SystemTrayIcon only (Windows/Linux only). And the SystemTrayIcon instance (DockIcon for MacOs) is retrieved from NativeApplication.nativeApplication.icon.
So this is where you should attach your event listener, after specifying the tray icon graphic:
var sti:SystemTrayIcon = NativeApplication.nativeApplication.icon as SystemTrayIcon;
// Specifying an icon is obligatory on Windows - MacOs has a default icon
sti.bitmaps = [new IconAsset()]; // IconAsset = Embedded picture
sti.addEventListener(ScreenMouseEvent.CLICK, mouseClick);
Note that the resultant screenX and screenY properties of ScreenMouseEvent are confined within the icon area in the tray and not the whole desktop screen (Not surprisingly, since this is where you added the event in the first place).

Resources