headers
Brion Vibber | 18 Sep 2012 09:31
Picon
Favicon
Gravatar

Initial stab at responsive images for high-density displays

How to load up high-resolution imagery on high-density displays has been an
open question for a while; we've wanted this for the mobile web site since
the Nexus One and Droid brought 1.5x, and the iPhone 4 brought 2.0x density
displays to the mobile world a couple years back.

More recently, tablets and a few laptops are bringing 1.5x and 2.0x density
displays too, such as the new Retina iPad and MacBook Pro.

A properly responsive site should be able to detect when it's running on
such a display and load higher-density image assets automatically...

Here's my first stab:
https://bugzilla.wikimedia.org/show_bug.cgi?id=36198#c6
https://gerrit.wikimedia.org/r/#/c/24115/

* adds $wgResponsiveImages setting, defaulting to true, to enable the
feature
* adds jquery.hidpi plugin to check window.devicePixelRatio and replace
images with data-src-1-5 or data-src-2-0 depending on the ratio
* adds mediawiki.hidpi RL script to trigger hidpi loads after main images
load
* renders images from wiki image & thumb links at 1.5x and 2.0x and
includes data-src-1-5 and data-src-2-0 attributes with the targets

Note that this is a work in progress. There will be places where this
doesn't yet work which output their imgs differently. If moving from a low
to high-DPI screen on a MacBook Pro Retina display, you won't see images
load until you reload.

Confirmed basic images and thumbs in wikitext appear to work in Safari 6 on
(Continue reading)

Permalink | Reply |
headers
K. Peachey | 18 Sep 2012 09:56
Picon
Gravatar

Re: Initial stab at responsive images for high-density displays

On Tue, Sep 18, 2012 at 5:31 PM, Brion Vibber <brion <at> pobox.com> wrote:
> * adds $wgResponsiveImages setting, defaulting to true, to enable the
> feature

Shouldn't experimental features be defaulting to false till it gets
tested a bit?
Permalink | Reply |
headers
Daniel Friesen | 18 Sep 2012 10:12
Gravatar

Re: Initial stab at responsive images for high-density displays

On Tue, 18 Sep 2012 00:56:46 -0700, K. Peachey <p858snake <at> gmail.com> wrote:

> On Tue, Sep 18, 2012 at 5:31 PM, Brion Vibber <brion <at> pobox.com> wrote:
>> * adds $wgResponsiveImages setting, defaulting to true, to enable the
>> feature
>
> Shouldn't experimental features be defaulting to false till it gets
> tested a bit?

I did that with the non-experimental combined login link that managed to  
make it's way into 1.19. And when I later set it to on-by-default in 1.20  
someone finally noticed that the code already released into 1.19 had an  
important bug in it.

Experimental features don't get properly tested. And they end up remaining  
buggy and experimental for several more releases than they should be.

--

-- 
~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://daniel.friesen.name]
Permalink | Reply |
headers
Daniel Friesen | 18 Sep 2012 10:46
Gravatar

Re: Initial stab at responsive images for high-density displays

On Tue, 18 Sep 2012 00:31:49 -0700, Brion Vibber <brion <at> pobox.com> wrote:

> How to load up high-resolution imagery on high-density displays has been  
> an
> open question for a while; we've wanted this for the mobile web site  
> since
> the Nexus One and Droid brought 1.5x, and the iPhone 4 brought 2.0x  
> density
> displays to the mobile world a couple years back.
>
> More recently, tablets and a few laptops are bringing 1.5x and 2.0x  
> density
> displays too, such as the new Retina iPad and MacBook Pro.
>
> A properly responsive site should be able to detect when it's running on
> such a display and load higher-density image assets automatically...
>
> Here's my first stab:
> https://bugzilla.wikimedia.org/show_bug.cgi?id=36198#c6
> https://gerrit.wikimedia.org/r/#/c/24115/
>
> * adds $wgResponsiveImages setting, defaulting to true, to enable the
> feature
> * adds jquery.hidpi plugin to check window.devicePixelRatio and replace
> images with data-src-1-5 or data-src-2-0 depending on the ratio
> * adds mediawiki.hidpi RL script to trigger hidpi loads after main images
> load
> * renders images from wiki image & thumb links at 1.5x and 2.0x and
> includes data-src-1-5 and data-src-2-0 attributes with the targets
(Continue reading)

Permalink | Reply |
headers
Antoine Musso | 18 Sep 2012 15:23
Picon
Favicon

Re: Initial stab at responsive images for high-density displays

Le 18/09/12 09:31, Brion Vibber a écrit :
> More recently, tablets and a few laptops are bringing 1.5x and 2.0x density
> displays too, such as the new Retina iPad and MacBook Pro.

Please excuse my noobiness, but what 1.5x / 2.0x densities are referring
to? IIRC most computers used 72dpi and Microsoft as used 96dpi.

--

-- 
Antoine "hashar" Musso
Permalink | Reply |
headers
Dan Andreescu | 18 Sep 2012 16:05
Picon

Re: Initial stab at responsive images for high-density displays

DPI started to go a little crazy when mobile devices were introduced.  This
is a good history:

http://www.html5rocks.com/en/mobile/high-dpi/

And you can see that iDevices have screens with varied ppi:

http://stackoverflow.com/questions/1365112/what-dpi-resolution-is-used-for-an-iphone-app

As for Daniel's desire for bliss, I think the html5rocks article does a
good job of showing how to use srcset and build safe fallbacks for it.  I
feel like that's the right approach because as browsers mature we can just
turn off the fallback.

On Tue, Sep 18, 2012 at 9:23 AM, Antoine Musso <hashar+wmf <at> free.fr> wrote:

> Le 18/09/12 09:31, Brion Vibber a écrit :
> > More recently, tablets and a few laptops are bringing 1.5x and 2.0x
> density
> > displays too, such as the new Retina iPad and MacBook Pro.
>
> Please excuse my noobiness, but what 1.5x / 2.0x densities are referring
> to? IIRC most computers used 72dpi and Microsoft as used 96dpi.
>
>
> --
> Antoine "hashar" Musso
>
>
> _______________________________________________
(Continue reading)

Permalink | Reply |
headers
Jon Robson | 18 Sep 2012 17:47
Picon
Gravatar

Re: Initial stab at responsive images for high-density displays

Awesome!
Correct me if I'm wrong but the way this is currently written an image for
foo.jpg will first load foo.jpg then replace the src attribute for this
element then load the image foo-2.0.jpg ?

In which case we probably need to think more abut minimising this initial
load. I'd suggest not setting the src attribute and using a noscript tag
repeating the same html. We might even be able to write javascript that
parses the noscript tag and creates the new image above it...

Another common trick I've seen is the browser via javascript reporting that
they now have a 2.0x density display and then the php script serving these
as the default in future.

With regards to Daniel's email it might be worth supporting a polyfill such
as [1]. I'm still anxious that since no browsers have implemented it there
might be a few minor changes and I think working with an established
library will keep us up to date.

I think we should be defaulting to true. I agree with Daniel that
experimental features never seem to enabled and don't get properly tested.
This should be the norm. We should be bold! :)

[1] https://github.com/scottjehl/picturefill
On Tue, Sep 18, 2012 at 12:31 AM, Brion Vibber <brion <at> pobox.com> wrote:

> How to load up high-resolution imagery on high-density displays has been an
> open question for a while; we've wanted this for the mobile web site since
> the Nexus One and Droid brought 1.5x, and the iPhone 4 brought 2.0x density
> displays to the mobile world a couple years back.
(Continue reading)

Permalink | Reply |
headers
Trevor Parscal | 18 Sep 2012 18:38
Picon
Gravatar

Re: Initial stab at responsive images for high-density displays

In VisualEditor we ended up putting all CSS rules that include images in
*.icons-raster.css and *.icons-vector.css files, which are loaded
dynamically based on the window.devicePixelRatio property.

It has it's flaws, but the good thing is that it spares the device from
loading both versions. I dislike the approach we are using and am open
minded to alternatives, but any solution that makes a client load both
isn't really going to work.

- Trevor

On Tue, Sep 18, 2012 at 8:47 AM, Jon Robson <jdlrobson <at> gmail.com> wrote:

> Awesome!
> Correct me if I'm wrong but the way this is currently written an image for
> foo.jpg will first load foo.jpg then replace the src attribute for this
> element then load the image foo-2.0.jpg ?
>
> In which case we probably need to think more abut minimising this initial
> load. I'd suggest not setting the src attribute and using a noscript tag
> repeating the same html. We might even be able to write javascript that
> parses the noscript tag and creates the new image above it...
>
> Another common trick I've seen is the browser via javascript reporting that
> they now have a 2.0x density display and then the php script serving these
> as the default in future.
>
> With regards to Daniel's email it might be worth supporting a polyfill such
> as [1]. I'm still anxious that since no browsers have implemented it there
> might be a few minor changes and I think working with an established
(Continue reading)

Permalink | Reply |
headers
Brion Vibber | 18 Sep 2012 19:06
Picon
Favicon
Gravatar

Re: Initial stab at responsive images for high-density displays

On Tue, Sep 18, 2012 at 9:38 AM, Trevor Parscal <tparscal <at> wikimedia.org>wrote:

> In VisualEditor we ended up putting all CSS rules that include images in
> *.icons-raster.css and *.icons-vector.css files, which are loaded
> dynamically based on the window.devicePixelRatio property.
>
> It has it's flaws, but the good thing is that it spares the device from
> loading both versions. I dislike the approach we are using and am open
> minded to alternatives, but any solution that makes a client load both
> isn't really going to work.
>

It's great to avoid loading both when you can, and with CSS icons that's
relatively doable.

For what it's worth though, a (different) load-both scenario is in actual
use on http://www.apple.com/

It's not perfect, but as a workaround until srcset is live I'm happy enough
with it. Think of it as progressive loading. ;)

-- brion
Permalink | Reply |
headers
Martijn Hoekstra | 18 Sep 2012 19:15
Picon

Re: Initial stab at responsive images for high-density displays

Sending more data to primarily empower mobile devices sounds rather
counter-intuitive
On Sep 18, 2012 7:06 PM, "Brion Vibber" <brion <at> pobox.com> wrote:

> On Tue, Sep 18, 2012 at 9:38 AM, Trevor Parscal <tparscal <at> wikimedia.org
> >wrote:
>
> > In VisualEditor we ended up putting all CSS rules that include images in
> > *.icons-raster.css and *.icons-vector.css files, which are loaded
> > dynamically based on the window.devicePixelRatio property.
> >
> > It has it's flaws, but the good thing is that it spares the device from
> > loading both versions. I dislike the approach we are using and am open
> > minded to alternatives, but any solution that makes a client load both
> > isn't really going to work.
> >
>
> It's great to avoid loading both when you can, and with CSS icons that's
> relatively doable.
>
> For what it's worth though, a (different) load-both scenario is in actual
> use on http://www.apple.com/
>
> It's not perfect, but as a workaround until srcset is live I'm happy enough
> with it. Think of it as progressive loading. ;)
>
> -- brion
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l <at> lists.wikimedia.org
(Continue reading)

Permalink | Reply |
headers
Brion Vibber | 18 Sep 2012 19:17
Picon
Favicon
Gravatar

Re: Initial stab at responsive images for high-density displays

On Tue, Sep 18, 2012 at 10:15 AM, Martijn Hoekstra <
martijnhoekstra <at> gmail.com> wrote:

> Sending more data to primarily empower mobile devices sounds rather
> counter-intuitive
>

Amount of data, and *when that data comes in*, are distinct things. One
other change we've got brewing up for the mobile site is to delay loading
of collapsed section content by separating it out to an API request. This
gets you loading the initial page *immediately*, then loads up the rest.
Same amount of data, but the page above the sections is live and visible
and interactive before you load the other stuff.

-- brion
Permalink | Reply |
headers
Trevor Parscal | 18 Sep 2012 19:19
Picon
Gravatar

Re: Initial stab at responsive images for high-density displays

It's important to separate supporting retina display mobile and desktop
devices. Apple's web site uses the load both method to show off the retina
display MacBook - which is more likely to have a faster internet connection
and is a more powerful machine in general.

- Trevor

On Tue, Sep 18, 2012 at 10:17 AM, Brion Vibber <brion <at> pobox.com> wrote:

> On Tue, Sep 18, 2012 at 10:15 AM, Martijn Hoekstra <
> martijnhoekstra <at> gmail.com> wrote:
>
> > Sending more data to primarily empower mobile devices sounds rather
> > counter-intuitive
> >
>
> Amount of data, and *when that data comes in*, are distinct things. One
> other change we've got brewing up for the mobile site is to delay loading
> of collapsed section content by separating it out to an API request. This
> gets you loading the initial page *immediately*, then loads up the rest.
> Same amount of data, but the page above the sections is live and visible
> and interactive before you load the other stuff.
>
> -- brion
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l <at> lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
(Continue reading)

Permalink | Reply |
headers
Brion Vibber | 18 Sep 2012 19:24
Picon
Favicon
Gravatar

Re: Initial stab at responsive images for high-density displays

On Tue, Sep 18, 2012 at 10:19 AM, Trevor Parscal <tparscal <at> wikimedia.org>wrote:

> It's important to separate supporting retina display mobile and desktop
> devices. Apple's web site uses the load both method to show off the retina
> display MacBook - which is more likely to have a faster internet connection
> and is a more powerful machine in general.
>

Same method works on the Retina iPad, which is similarly powered to a
late-model iPhone. iPads, iPods, and MacBooks can all be on mobile networks
or super-fast wifi networks, of course. There's not a good way to assume
connection speed from.... anything. :)

-- brion
Permalink | Reply |
headers
Martijn Hoekstra | 18 Sep 2012 19:22
Picon

Re: Initial stab at responsive images for high-density displays

I hate derailing threads, bit doesn't the mobile skin already do this? I
always notice fast loading of articles on my cell phone, and slow section
opening (I figured it was quite clever to defer loading of sections to a
later point in time. For a quick lookup on my cell phone I rarely need to
load all sections anyway)
On Sep 18, 2012 7:17 PM, "Brion Vibber" <brion <at> pobox.com> wrote:

> On Tue, Sep 18, 2012 at 10:15 AM, Martijn Hoekstra <
> martijnhoekstra <at> gmail.com> wrote:
>
> > Sending more data to primarily empower mobile devices sounds rather
> > counter-intuitive
> >
>
> Amount of data, and *when that data comes in*, are distinct things. One
> other change we've got brewing up for the mobile site is to delay loading
> of collapsed section content by separating it out to an API request. This
> gets you loading the initial page *immediately*, then loads up the rest.
> Same amount of data, but the page above the sections is live and visible
> and interactive before you load the other stuff.
>
> -- brion
> _______________________________________________
> Wikitech-l mailing list
> Wikitech-l <at> lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
Permalink | Reply |
headers
Brion Vibber | 18 Sep 2012 19:23
Picon
Favicon
Gravatar

Re: Initial stab at responsive images for high-density displays

On Tue, Sep 18, 2012 at 10:22 AM, Martijn Hoekstra <
martijnhoekstra <at> gmail.com> wrote:

> I hate derailing threads, bit doesn't the mobile skin already do this

Nope.

> I
> always notice fast loading of articles on my cell phone, and slow section
> opening (I figured it was quite clever to defer loading of sections to a
> later point in time. For a quick lookup on my cell phone I rarely need to
> load all sections anyway)
>

The whole page is loaded, then elements are collapsed. This means
relatively slow loading, including loading of images in collapsed sections.

That's why we want to be clever and add delayed loading of those sections.
:)

-- brion
Permalink | Reply |
headers
Krinkle | 20 Sep 2012 21:42
Picon
Gravatar

Re: Initial stab at responsive images for high-density displays

On Sep 18, 2012, at 5:47 PM, Jon Robson <jdlrobson <at> gmail.com> wrote:

> Awesome!
> Correct me if I'm wrong but the way this is currently written an image for
> foo.jpg will first load foo.jpg then replace the src attribute for this
> element then load the image foo-2.0.jpg ?
> 

It did that because the javascript function was hooked on window.load, which by
design does not fire until all images are downloaded.

The patch [1] has been revised and now fires on document ready, which should be
early enough to not waste much bandwidth.

I suggest we built-upon or write or own module further and integrate the
"lazy-load" principle. In other words, on document ready fix the images above
the fold, which may or may not have started downloading yet.

Then cancel the rest and set those appropriately just before they come into
view. That saves bandwidth in general (by not loading images when they are not
visible), and makes sure to download the right image based on the environment at
that point in time.

When a standard for srcset (or whatever it will be called) is ready and actually
implemented in some browser we could also opt to keep it without javascript.

Assuming plans won't get worse, the standard will include a natural fallback by
storing the 1-0 ratio image in the src attribute. Which is what we'd want on
older browsers/devices anyway.
(Continue reading)

Permalink | Reply |
headers
Brion Vibber | 24 Sep 2012 04:59
Picon
Favicon
Gravatar

Re: Initial stab at responsive images for high-density displays

On Thu, Sep 20, 2012 at 12:42 PM, Krinkle <krinklemail <at> gmail.com> wrote:

> I suggest we built-upon or write or own module further and integrate the
> "lazy-load" principle. In other words, on document ready fix the images
> above
> the fold, which may or may not have started downloading yet.
>
> Then cancel the rest and set those appropriately just before they come into
> view. That saves bandwidth in general (by not loading images when they are
> not
> visible), and makes sure to download the right image based on the
> environment at
> that point in time.
>

Hmm... there are problems with this such as printing compatibility, but
this will also be combated in the future (and on mobile) with collapsible
sections and such. Worth considering...

> When a standard for srcset (or whatever it will be called) is ready and
> actually
> implemented in some browser we could also opt to keep it without
> javascript.
>
> Assuming plans won't get worse, the standard will include a natural
> fallback by
> storing the 1-0 ratio image in the src attribute. Which is what we'd want
> on
> older browsers/devices anyway.
>
(Continue reading)

Permalink | Reply |
headers
Brion Vibber | 28 Sep 2012 00:45
Picon
Favicon
Gravatar

Re: Initial stab at responsive images for high-density displays

On Sun, Sep 23, 2012 at 7:59 PM, Brion Vibber <brion <at> pobox.com> wrote:

> I've updated the patch to use the 'srcset' attribute as defined in the
> current HTML 5 working group version, only using the density options and
> not the width/height options:
>
> http://www.whatwg.org/specs/web-apps/current-work/multipage/embedded-content-1.html#attr-img-srcset
>
> Patch in gerrit:
> https://gerrit.wikimedia.org/r/#/c/24115/ (core)
> https://gerrit.wikimedia.org/r/#/c/24147/ (MobileFrontend)
>

Patch has been updated to:
* include QUnit tests for the srcset matching
* match "next-highest density" if there's not an exact match available

This latter fixes Opera Mobile on my Galaxy Nexus (where devicePixelRatio
reports 2.25 rather than the expected 2) and BlackBerry 10 developer alpha
device (where it reports slightly less than 2.25). Should help with some
other funky configurations where a non-integral zoom is reported as well.

Currently Firefox for Android doesn't report window.devicePixelRatio, I may
add another special case, should be able to use media queries.

Opera Mobile works, but Opera Mini does not -- at least because we don't
serve it jQuery etc. :)

You can see a live demo of this patch in action on these test articles:
* http://leuksman.com/mw/index.php/San_Francisco
(Continue reading)

Permalink | Reply |
headers
Brion Vibber | 18 Sep 2012 20:37
Picon
Favicon
Gravatar

Re: Initial stab at responsive images for high-density displays

On Tue, Sep 18, 2012 at 12:31 AM, Brion Vibber <brion <at> pobox.com> wrote:

> Here's my first stab:
> https://bugzilla.wikimedia.org/show_bug.cgi?id=36198#c6
> https://gerrit.wikimedia.org/r/#/c/24115/
>

I've made a couple changes in patchset 2:

* fix for images that specify width but not height (whoops! this broke
on... a lot of images in my real-world test case "San Francisco" enwiki
article)
* apply image 'src' replacements immediately on DOM-ready instead of
waiting for window.onload

This should prevent some, but not necessarily all "double loading". Modern
browsers are pretty aggressive about preloading images, and at least some
may start loading at 1.0x resolution before they get replaced.

Jon's suggestions of using a blank src and a <noscript> are doable but...
it gets ugly fast to make it work for non-JS cases cleanly. Not sure if
it's worth going down that road, but willing to try. :)

[Note that it still fails jenkins tests because test cases haven't been
updated.]

-- brion
Permalink | Reply |

Gmane