Guide

Citrix Receiver for Linux on Raspberry Pi 2 using Raspbian Wheezy

Martin 50 Comments , , , , , , , , , ,

Pi_Receiver_Board_300Muhammad @Citrix initially created a proof of concept of the Citrix Receiver for Linux for the Raspberry Pi, it sparked a lot of interest as can be seen on the comments on the blog series he posted (Part 1, Part 2, Part 3). The idea of having a Raspberry Pi out performing many of the existing (and more expensive) thin clients at an amazingly low cost was of course going to be of interest.

When the Pi 2 came along with the potential for performance to take another step up was clear. As can be seen from the comments Citrix is working with the Raspberry Pi Foundation and 3rd party vendors looking to create add management capabilities to a Raspberry Pi thin client, e.g: ThinlinX.

This post covers many of the points covered in the comments on Muhammad’s blog posts. But goes further, by providing the individual steps necessary to set up and optimise the Citrix Receiver, ready to connect to a Citrix XenApp server. This guide is based on starting from a completely clean image of Raspbian Wheezy on my Pi 2.

UPDATE: I’ve also posted an updated guide for use on the newer Raspbian Jessie release.

The Goal

Allow the standard Citrix Receiver for Linux to run unmodified on a Raspberry Pi 2. e.g. Linux Receiver for ARMHF

The Problem

The Citrix Receiver for Linux has a dependency on glibc6 version 2.15-0 or later. Raspbian Wheezy currently ships with 2.13-38 as can be seen by checking the output of ldd –version:

pi@raspberrypi ~ $ ldd --version
ldd (Debian EGLIBC 2.13-38+rpi2+deb7u8) 2.13

Later versions of glibc aren’t available in the Debian Wheezy branch. They are however available in later Debian release branches. If you do manage to get it installed there are still changes you need to make to improve the performance to a usable level.

The Solution

As of today (1 Aug 2015) there isn’t a ready packaged solution. When Raspbian updates to a Debian “Jessie” derivative I’ll review which steps need to change. For example Jessie includes the required version of glibc6.

The Workaround

We also need to make some changes to the Citrix Receiver and the Raspberry Pi’s configuration.

Step 1: Update the glibc to version >= 2.15

The OS needs to be updated to add a newer version of glibc, this updated version isn’t available in the stable branch, instead we need to get this from the unstable (sid) branch.

1.1 Update package sources to include unstable packages

Edit files using vi, nano or whatever you like, such as Sublime Text on my Windows PC, see how here.

  • sudo nano /etc/apt/sources.list
  • Add an # to the start of the line current “main” repository to comment it out:
    • #deb http://mirrordirector.raspbian.org/raspbian/ wheezy main contrib non-free rpi
  • Add a new line:
    • deb http://ftp.us.debian.org/debian sid main
  • Save and then exit your editor
1.2 Install the updated glibc6 from the unstable branch
  • sudo apt-get update && sudo apt-get -t sid install libc6
  • When prompted update the packages, I had 14 to upgrade.
  • When prompted accept to install the packages that can’t be authenticated.
  • When prompted let it restart the services
  • Confirm the version of glibc6 has been updated to a version > 2.15:
    • ldd --version should report 2.19 i.e:
pi@raspberrypi ~ $ ldd --version
ldd (Debian GLIBC 2.19-19) 2.19
Copyright (C) 2014 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

1.3 Change packages sources back to stable branch

To avoid accidentally updating the whole distribution to unstable packages, change the sources back

  • Undo the changes from step 1.1. Edit /etc/apt/sources.list un-comment the original package repository and comment out the unstable repository.
  • Refresh the packages with the original repository: sudo apt-get update

Step 2: Download and Install the Citrix Receiver

The Citrix Receiver has a number of package dependencies, some required some recommended. I’d recommend you use gdebi to install the packages as it will find and install the dependent packages and therefore simplify the setup. So if you don’t have it installed already run the command: sudo apt-get install gdebi (Thanks to Ben Waine for making me aware of gdebi).

Also ensure you download the ARMHF packages and not the ARMEL versions.

  • Download the Debian (.deb) packages of Citrix Receiver for Linux (ARMHF) and the (optional) USB Support Package from citrix.com. (Current links are to the 13.2 client, always best to check if there is a newer version).
  • Install the Receiver: sudo gdebi icaclient_13.2.0.322243_armhf.deb
  • (optional) Install the USB Support package: sudo gdebi ctxusb_2.5.322243_armhf.deb

Step 3: Test the Client

You’ll need to have GUI running if you haven’t already. If at a terminal prompt, enter the command startx to load the GUI

3.1 Configure the Citrix Receiver for Linux
  • Launch Citrix Receiver, either from the desktop menu item or run: /opt/Citrix/ICAClient/selfservice & 
  • Configure your connection. If using a StoreFront store you’ll need to use HTTPS and ensure you have the necessary pem formatted certificate download to the keystore/castore. For simplicity I’m going to use the PNAgent site on my XenApp 7.6 deployment.
    • The address of which I enter as:
    • If you are looking for your PNAgent site URL, you’ll find it by running Citrix Studio, Open the Citrix StoreFront console node: Stores -> “Configure XenApp Services Support”. The dialog that appears will show if PNAgent is enabled and the URL.
  • Add one of your published resources (Applications or Desktops)
  • Connect. You should hopefully have a successful session.

 

Step 4: Optimising Performance

4.1. Measure Performance

You can measure some elements of session performance my examining information from Citrix HDX Monitor, which can be downloaded and run within the session. If you run it you’ll see something like the images below (though these are taken from a Windows Client):

HDXMonitor_HomeHDXMonitor_Thinwire_Advanced

 

An alternative to using HDX Monitor, the Receiver for Linux offers some client side metrics which can be enabled via parameters for WFICA_OPTS (see the section “Monitor real-time performance” in the Citrix Receiver for Linux OEM’s Reference Guide for more information on what each of these values do). Using these options we can assess the impact of the suggested changes. Before running the Citrix Receiver, run run command:  export WFICA_OPTS="-rm Dcdjfresot"

Benchmark_Pi2For my test I connected to a XenApp 7.6 Desktop at 1600×1200 and ran a 720p video using VLC. The frame rates and CPU usage values quoted are the average during video playback. The initial rate achieved before any (ok I’d fixed the CPU Governor to ondemand rather than powersave) optimisation was 6.8 FPS @ 45% CPU usage. After all the optimisations below are applied, including the Pi2 preset overclock; there was a 178% improvement to 19 FPS @ 34% CPU usage.

 

4.2 Check your PSU

If you ever see a small rainbow coloured square on the top right of your display, this is an indication that your Pi is not receiving sufficient power. You should review the PSU are using ideally you want a PSU with a 5V 2A (2000mA) output. The actual power draw of your Pi will be dependent on devices you have attached. In addition to the rainbow square, the Pi will not be able to run the CPU at maximum power.

4.3 CPU Governor (Pi Setting)

This controls the one of the power management features of the CPU, see: cpu-freq documentation.

  • Check the current mode of your CPU: cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
  • My output: powersave
  • It appears to get the default to be ondemand, we simply need to update the Pi with the latest updates:
    • sudo apt-get update && sudo apt-get upgrade
  • Reboot your Pi (sudo reboot)
  • Login and repeat the 1st command to confirm the scaling_governor is now: ondemand

Performance: 6.8 FPS @ 45% CPU – Using Citrix Compatibility Graphics mode

4.4 Set the Framebuffer depth (Pi Setting)

  • Edit the file /boot/config.txt
  • You can place this change anywhere in the file, but it makes sense beneath the entries for framebuffer_height, which are commented out by default
  • Add the line: framebuffer_depth=32
    • Note: According to elinux.org setting the framebuffer_depth > 16 will also need: framebuffer_ignore_alpha=1. My basic testing didn’t observe any difference, so perhaps this has been resolved in more recent releases? 
  • Save and exit
  • Reboot your Pi (sudo reboot)

Performance: 8.9 FPS (+30%) @ 48% CPU (+5%) – Using Citrix H264 Graphics mode

4.5 Switch to accelerated jpeg libraries (libjpeg62-turbo) (Software packages)

The ARM processor in the Pi 2 supports some extensions to accelerate certain operations, called NEON. Intel has MMX and SSE2. Using these extensions libjpeg-turbo claims a 2x-4x speed improvement. The Citrix Receiver configuration needs to be modified a little to take advantage of this. Once again we have to use the unstable branch to access the required packages.

  • Switch package sources to the unstable branch. Follow the steps in 1.1 (above)
  • Install the libjpeg76-turbo package: sudo apt-get update && sudo apt-get install libjpeg62-turbo
  • Repeat step 1.5 from above to revert the changes to the repository locations
  • The Citrix receiver works through possible jpeg decoders it can use. On my Pi 2 it was using ctxjpeg_fb_8.so as /usr/lib/arm-linux-gnueabihf/libjpeg.so.8 was present, however this isn’t the accelerated version. So we need to rename this so that the Citrix Receiver doesn’t try and use the version 8 fallback.
    • sudo mv /opt/Citrix/ICAClient/lib/ctxjpeg_fb_8.so /opt/Citrix/ICAClient/lib/ctxjpeg_fb_8.so.DO_NOT_USE
  • Now the Citrix Receiver will use ctxjpeg_fb.so which is linked to the newly installed libjpeg62-turbo library.

Performance: No change as expected in H264 mode.

4.6 Disable H264 graphics mode (Citrix Setting)

A side effect of increasing the size of the frame buffer (step 5.1), is that by default connections will now switch to H264 mode. However this won’t give the best experience on a Raspberry Pi, so we’ll want to disable this for all connections.

  • Edit: /opt/Citrix/ICAClient/config/module.ini
  • Under the section for: [WFClient]
  • Add the line: H264Enabled=False
  • Save the file

Performance: 14 FPS (+105%) @ 35% CPU (-22%) – Using Citrix Compatibility Graphics mode

4.7 Disable Multimedia Redirection (Citrix Setting)

When you make a connection to your XenApp/XenDesktop Server you may want to try playing some video content. Citrix has a feature (HDX MediaStream aka RAVE) which can redirect video to be played using client side resources rather than playing it server side. The Pi, however doesn’t work well with this. So we’ll want to render our video server side.

  • Edit: /opt/Citrix/ICAClient/config/module.ini
  • Under the section for: [ICA 3.0]
  • Edit the line for: VirtualDriver =
  • Remove the entry for: MultiMedia
  • Save the file

Performance: No change as test uses server side rendered video.

4.8 Implement Thinwire+ (aka Project Snowball) (Citrix additional software)

The Thinwire+ feature is now available in XenApp/XenDesktop 7.6 Feature Pack 3, see: https://www.citrix.com/blogs/2015/10/01/feature-pack-3-for-xenapp-and-xendesktop-7-6-is-now-available/

Performance: 16.7 FPS (+145%) @ 34% CPU (-25%) – Using Citrix Compatibility Graphics mode with snowball private

Step 5: Further Performance (optional)

The raspi-config utility provides some simple steps to try out various levels over overclocking your Pi. Online you can buy heatsinks and cooling fans if you feel the need to push the limits of your Pi and go beyond some of the standard overclock settings. However if you push the limits it tends to lead to instability. Testing with Pi2 preset I didn’t have any issues with stability.

5.1 Setting overclock setting to the “Pi2” preset

raspi-config-overclock-settings

  • Increases the maximum clock speed (arm_freq) by 11% to 1000MHz.(The min remains 600Mhz)
  • Increases the core (GPU) frequency (core_freq) by 100% to 500MHz
  • Increases the memory frequency (sdram_freq) by 25% to 500MHz
  • Increases the over_voltage to 2, effectively providing an additional 2 x 0.025V to the base 1.2V CPU voltage. This is generally needed to balance the increase power draw from increasing clock frequencies.
Final Performance: 19 FPS (+178%) @ 34% CPU (-25%) – Using Citrix Compatibility Graphics mode with snowball private

Pi2_Peformance_Client_FPS

Acknowledgements

I must give huge thanks to Muhammad Dawood (LinkedIn / Citrix-Blogs). Not only for his initial efforts to port the Linux client to the Raspberry Pi 1 but specifically for working with me to determine the various changes/tweaks necessary to get the best out of the Receiver running on the Pi 2 using Raspbian Wheezy.

Secondly, I must thank Tobias Kreidl (LinkedIn) who responded to some of my very early posts on this blog. His interest in a Citrix Receiver for Pi 2 sparked my realisation for the need to create a step by step guide.

Additional Information

You May Also Like

Unifi ssh terminal showing snmpwalk

Enabling SNMP on Ubiquiti UDM Pro

Martin 59
Pi4 Hot Case

Raspberry Pi 4 Hot new release – Too hot to use enclosed

Martin 3

Buying a Raspberry Pi 2 – An unexpected decision

Martin 3

50 thoughts on “Citrix Receiver for Linux on Raspberry Pi 2 using Raspbian Wheezy”

  • This is an excellent, incredibly detailed posting, so many thanks for all the time and effort you put into this, Martin!

    Reply

  • A question: There have been some discussions regarding adequate power supply amperage/Wattage. What power supply specifications are you using in particular to be able to support the overclocking?

    Reply

    • I’m using a 5V 2A power supply and haven’t had any issues running with the Pi2 Preset, leaving the video looping for hours. With 2 USB ports populated with Wi-Fi module and Bluetooth modules.

      Reply

  • With trying to get H.264 working on a Pi 2B, it looks like there are issues with compatibility with the VC4 libraries and that in turn appears to be a Broadcom library issue which could probably be solved if built using the right Debian ARMHF configuration. Yi Lu posted an update at https://github.com/luyi1888/ctxh264_pi regarding a Platform Optimization SDK for H.264 and that is on the right track. Even better would be of course moving away from the H.264 decoding dependency and shifting to Thinwire Plus support. Any efforts to get this rolling along faster (pun intended) and released would be great!

    Reply

  • Firstly to clarify, Thinwire Plus aka Snowball is available today. You just need to escalate to Citrix Support to obtain it. The next release public VDA release will include this support without the need to escalate to get it.

    For people who want to adopt the hardware accelerated H264 decoder Yi Lu looks to be on the right track. I was planning on working trying to get this working myself, so I’ll have to test out if Yi Lu has implemented this as I planned to. There will always be some sub-optimal elements in this mode. But I do wonder if there really is any benefit to supporting it. For now Thinwire plus looks good enough for most things.

    Reply
    • Anonymous
      8 August 2015 at 11:37 pm
      Permalink

      Martin, that is great news on Thinwire Plus, as its official release and support by Citrix was not clear/obvious. Perhaps holding a celebration would be a good idea? 🙂 Both protocols (Thinwire and H.265) have their place; it’s a matter of what performs better and also what is easier to implement, and perhaps even more important, what is easier to maintain down the road. Many thanks in any case for the updates and we’re all looking forward to the combined efforts to bring this all to fruition (again, pun intended).

      Reply

  • Thanks so much.

    For anyone with issues with certificates use this

    sudo ln -s /usr/share/ca-certificates/mozilla/* /opt/Citrix/ICAClient/keystore/cacerts/
    sudo c_rehash /opt/Citrix/ICAClient/keystore/cacerts/

    Reply

  • Updated page with details on how to get the Snowball aka Thinwire Plus Tech Preview build. I though it was only available via Escalation, however it turns out if you have a myCitrix logon you look to be able to get it. So try it, if you haven’t already.

    Reply

  • I’m not sure this is the latest version of Thinwire Plus. The one available on-line from the technical preview area at the myCitrix site is this:

    HDX Enhanced ThinWire Compatibility Mode Tech Preview
    May 8, 2015
    3.77MB – (.zip) Download

    This package includes the Admin Guide, prerequisite Hotfix, and new encoder that may be applied to VDA 7.6.
    MD5: e91f30d4909dabc17c2e105d7ef42615

    Thanks in advance for the clarification,
    –Tobias

    Reply

  • Hi Guys,

    we use the Raspi2 with Wheezy(also tested with jessie) with Citrix Receiver 13.2 as Thin Clients, but we have the Problem, that the Citrix Window just closes without a reason. No entrys in any logs. It happens completely randomly. It seems that nothing triggers the crash… We are using XenApp6.5 btw.

    Does anyone got a hint how to fix this?

    Reply

    • So to confirm it works and you can make connections but during a session it disappears. Does anything happen just before it disappears? For example freeze or get a bit less responsive?

      Reply
      • Anonymous
        28 September 2015 at 1:42 pm
        Permalink

        Hi,

        I have the same problem than Mattz.
        Do you have any idea to solve this problem ?

        Thks.

        Reply
  • Mike Birt
    28 August 2015 at 5:02 pm
    Permalink

    I am having trouble updating the ldd. i receive error “E: The value ‘sid’ is invalid for APT::Default-Release as such a release is not available in the sources”. i know my sources are updated and match letter for letter but what am i missing?

    thanks,

    Mike

    Reply

    • Hi Mike, I’m not near my Pi at the moment, so I can’t double validate this.

      After modifying /etc/sources/list you did run: sudo apt-get update
      Then when attempting to update libc6, you did include the -t sid option: sudo apt-get -t sid install libc6

      If all that is true, then can you confirm if you are working from a clean/fresh Raspbian Wheezy image, or some other image?

      Also what is the contents of: /etc/apt/apt.conf

      Reply

      • FYI The contents of my sources.list looks like this:
        deb http://mirrordirector.raspbian.org/raspbian/ wheezy main contrib non-free rpi
        # Uncomment line below then ‘apt-get update’ to enable ‘apt-get source’
        #deb-src http://archive.raspbian.org/raspbian/ wheezy main contrib non-free rpi
        #deb http://ftp.us.debian.org/debian sid main

        Showing the default sources uncommended and the test/sid one commented out. If I enable the sid source, and run apt-get update is works correctly.

        I also checked my Pi and I don’t have an /etc/apt/apt.conf file.

        The default sources file looks to be backed up as: /etc/apt/sources.list.d/raspi.list

        Reply
        • Mike Birt
          1 September 2015 at 5:46 pm
          Permalink

          still failing with “E: The value ‘sid’ is invalid for APT::Default-Release as such a release is not available in the sources”

          this is a fresh install from NOOBS.
          yes i ran the update
          yes i ran the “-t” option
          i added the src option in the list
          commented 2 of the 3 out at a time with failure
          ran the update with none of them commented out
          i have even tried running it with the ftp line in the list and all times have that same error.

          Reply

    • Can you confirm if you comment out the sid line, that if you run apt-get update followed by apt-get upgrade that the basics of package updating is working and it will update your clean NOOBs image.
      Can you also check if you do have an /etc/apt/apt.conf file as a little searching around indicated there may be a config file that can specify which streams updates can come from.

      Reply
      • Mike Birt
        2 September 2015 at 2:54 pm
        Permalink

        i love and hate linux! the problem was with my list in that i edited souces.list instead of sources.list

        after editing the correct list it worked

        Reply

  • Pingback: Understanding Citrix Ready Certification and Thin Clients | Virtually Visual

  • Tobias K
    8 September 2015 at 10:40 pm
    Permalink

    Any idea when a Jessie build will be available? Citrix Receiver 13.2.1 is now out.

    Reply

  • I believe it should be soon, but ensuring the stability, reliability and ease of use of Raspbian is critical to its release. I’m checking regularly and have a post drafted and ready to go once it’s released, as it simplifies the deployment a little.

    Reply

  • first thanks for your works! I’m using some raspy 2 with citrix receiver and all of your optimizations and vision is fluid.
    now I started using an usb plantronics headset with microphone to use our cisco ip comunication software, but audio is crackly and voice is not audible.
    can I do something to enhance it?

    Reply

    • By default your headset is probably being mapped using standard client audio mapping. If you run HDX monitor in your session you should be able to see the quality levels that applied to your connection. There are a number of policies that can be configured to control audio quality and the codecs being used. I’d suggest you give that a try first. If that doesn’t help, then trying the USB redirection might be worth a shot, though you’ll have to modify usb.conf to allow audio devices to be mapped through. Hope this helps, I’ll admit I’ve not tried audio devices yet with my Pi setup.

      Reply
  • SvenS
    15 September 2015 at 12:18 pm
    Permalink

    my connection always shows h264=off, i tried the precompiled version and also compiled on my own, any hints where to check for the cause?

    Reply

    • In /boot/config.txt have you set the framebuffer depth to 32 (or 24 as a min.). Also if you are trying to compile and use the H264 hardware encoder support, you’ll need to add gpu_mem=128 into the config file too.

      Thanks Martin

      Reply

  • nevermind, the framebuffer was not running in 32bit, the decode seems to be locked at 30fps, is there any known limitations?

    Reply

  • nope, we are not policy limited to 30FPs, we have a hdx 3d pro with grid setup, running fluid at 60fps on other devices (thinlinx nucs & windows receivers). we use pure h264 with zero compression applied. it does not seem to be a performance issue but a hardlock, see image for reference.

    i disabled flowcontrol on the endpoint and in the session itself.

    is there any way to check the load on the encoding soc?

    http://i.imgur.com/nAtNzDc.jpg

    Reply

    • looks like some kind of flow control is still running, we can get up to 47fps but the network io is pretty low, we had the same issue on a flow control enabled linux. we will dig into the configs again

      Reply

  • 21 fps without Snowball while running a youtube Video 🙂 nice!

    Reply

  • thanks martin, that seems to be the issue, sad but true.

    there is a nice reverse engineering github about the videocore iv from the bcm2835, same as the bcm2836 in the pi2

    https://github.com/hermanhermitage/videocoreiv
    https://github.com/hermanhermitage/videocoreiv/wiki/VideoCore-IV—BCM2835-Overview

    quote

    Video Encoder / Decoder

    1080p30 Full HD HP H.264 Video Encode/Decode
    The GPU can hardware decode H264, MPEG1/2/4, VC1, AVS, MJPG at 1080p30.

    so 720p at 60fps and 720p+ at 30fps.

    Reply

  • Hi guys,

    I’m currently operating with citrix receiver 13.2.1 on Rpi2 and Linux VDA on CentOS 6. I found one problem, that mouse cursor mapping is not working properly, better is to say is not working at all. When I’m trying citrix receiver 13.2.1 for x86_64 on my Debian with GNOME 3, all is working fine. So did you try combination of Rpi 2 and Linux VDA CentOS/RHEL 6?

    I tried already change X-window server to various types, but no joy. I tried also different graphical evironments – LXDE, XFCE, Ubuntu MATE, but still same problem. So, it means that problem seems to be in citrix receiver(ARM HF) itself.

    Thanks in advance guys..

    Reply

    • I’ve only tried on Raspbian. I briefly installed Ubuntu MATE but found the performance terrible, that was just of the Linux Desktop.
      What Citrix Receiver graphics mode are you trying to use with the Receiver running on CentOS? Legacy/Compatibility/H264?
      Have you confirmed it works correctly if you use a Raspbian image, just to rule out any hardware issues?

      Reply

  • Hi Martin,

    thanks for your reply. Yes I tried citrix receiver 13.2.1 on raspbian firstly, but when i wasn’t able to get it work(mouse cursor issue) then I tried alternatives like different x-window manager, Ubuntu MATE and so on. Behaviour is different for Windows VDI and Linux VDI. When I’m trying to connect from Rpi 2 client to Windows 2012 R2 all is working fine and cursor is visible correctly. But when I’m connecting to Linux VDI(Linux VDA 1.0 installed on CentOS 6) then I haven’t mouse cursor visible.

    Now I’ll try disable H264 graphics mode and enable logging to file for advance troubleshooting. I’ll let you know if this solve my problem or not.

    Thanks for cooperation,

    Josef

    Reply

    • Thanks for the extra information, so it’s something specific to a combination of LinuxVDA and the Client running on RPi. I’ll ask around to see if anyone knows anything about this.

      Reply

  • Thanks for your effort posting this!

    Reply

  • Hi guys,

    solution for my problem with cursor rendering is:

    In /home/USER/.ICAClient add the following line in your wfclient.ini:
    DisableXRender=1

    Cheers

    Reply

  • I just wanted to thank you for your effort creating this detailed guide.
    Keep up the good work!

    Reply

  • Hi Martin,

    Any news with a Jessie update? I have found that a base Jessie install, followed by a gdebi installation, Pi boots fine. As soon as I install Citrix 13.2.1 and its dependents the Pi won’t start an X session, I just get a flashing cursor post boot up.

    Reply

  • Pingback: Citrix Receiver on Raspberry Pi -Raspbian Jessie | Infrastructure:land

  • Pingback: Virtualisation de bureau (VDI) sur Raspberry Pi 2 | Portfolio-Noiret

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.