color/symbol mapping breaks `transforms` filter

[mevers]

Issue

transforms filter plus color/symbol mapping seems to break the transforms filter.

Reprex

Sample data

df <- data.frame(
    x = c(1:10, 1:10),
    y = c(1:10, 10:1),
    z = c(rep("up", 10), rep("down", 10)))

The idea is to only show data where z = "up" through a transforms filter.

Case 1: transforms filter without color mapping

This works

plot_ly(
    df,
    x = ~x, y = ~y, text = ~z,
    type = "scatter", mode = "markers+text",
    marker = list(size = 10, opacity = 0.8),
    transforms = list(
        list(
            type = "filter",
            target = ~z,
            operation = "=",
            value = "up")))

Case 2: transforms filter with color mapping

This does not work

plot_ly(
    df,
    x = ~x, y = ~y, text = ~z, color = ~z,
    type = "scatter", mode = "markers+text",
    marker = list(size = 10, opacity = 0.8),
    transforms = list(
        list(
            type = "filter",
            target = ~z,
            operation = "=",
            value = "up")))

Summary

In both cases the transforms filter is the same and should select only those entries where z = "up"; however, when a color (or symbol) mapping is added, for some reason only entries where z = "down" are shown.

[mevers]

This issue was originally reported on Stack Overflow by user @FKneip: https://stackoverflow.com/questions/55251747/plot-ly-with-transforms-and-color-mapping

SO user @RolandASc traced this back to an inconsistency with sorting in plotly_build:

Looking at plotly:::plotly_build.plotly, we can see that sorting is applied in certain cases, e.g. when working with groups, or discrete colors or symbols. This sorting however does not include the transforms/filter, therefore the two things get out of sync.

A workaround is to sort the data first according to the factor that is to be used in the colour or symbol mapping.

[cpsievert]

Oof, yea, this will be difficult to fix, but you can workaround the issue by using customdata instead of transforms[i].target, like this:

plot_ly(
  df,
  x = ~x, y = ~y, text = ~z, color = ~z, customdata = ~z,
  type = "scatter", mode = "markers+text",
  marker = list(size = 10, opacity = 0.8),
  transforms = list(
    list(
      type = "filter",
      target = "customdata",
      operation = "=",
      value = "up")))

[mevers]

Hi @cpsievert

I'm hoping you'd have some tips on how to deal with what I think is a related issue (if you think this is unrelated I'm happy to move it elsewhere). As per your earlier advice, I've been using customdata for the filtering.

Things seem to go a bit wonky, when using group_by plus color and customdata (for dropdown filtering).

Here's a reproducible code example:

# Sample data
set.seed(2020)
df <- replicate(2,
    data.frame(
        x = rep(1:4, 4),
        y = cumsum(rnorm(4 * 4)),
        id = rep(1:2, each = 4),
        col = rep(LETTERS[1:2], each = 8)),
    simplify = FALSE) %>%
    setNames(c("One", "Two")) %>%
    bind_rows(.id = "sel")

plotly code:

df %>%
    group_by(id) %>%
    plot_ly(
        x = ~x, y = ~y, color = ~col, customdata = ~sel,
        transforms = list(
            list(
                type = "filter",
                target = "customdata",
                operation = "=",
                value = "One"))) %>%
    add_lines() %>%
    add_markers()

producing

This is almost what I expect, except for the lines connecting measurements from the same colour but with a different id (they shouldn't be connected).

For comparison, what I was hoping for was a plot similar to the top left panel of the static ggplot2 version, i.e. without lines connecting points with different ids.

ggplot(df, aes(x, y, colour = col, group = interaction(id, col))) +
    geom_line() +
    geom_point() +
    facet_wrap(~ sel)