44 posts

Aug 22 2021

GSoC 2021 With XWiki For XWiki-AWS Integration

gsoclogo.png    xwiki logo.png

About me

I am Sanchita Singh, an undergraduate student at Indian Institute of Technology Kharagpur. This was my first time participating for GSoC and I got the opportunity to work under XWiki for developing an easy installation method of production instance of XWiki in AWS account.

Description

The project was aimed to increase the number of active installs in XWiki by providing the end-users with the flexibility to use Cloud Computing or services provided by Amazon Web Services (AWS). There are Three Different parts of this project, three different ways of installing XWiki.

  • XWiki installation using Cloudformtion template (This method is for non-developers who want an easy 1-click installation solution, where you can install your production-ready XWiki in a couple of clicks on AWS)
  • XWiki installation using CDK code (this installation method is also straightforward, but This method is specifically for developers as they will be able to tweak the configurations of the system as per their need since CDK provides a high-level interface where developers can use high-level languages to interact with lower-level Cfn classes)
  • XWiki installation by using AMI on the marketplace (Since marketplace supports only AMI-based templates, we could not use the system design we created for Cloudformation based and CDK-based installation and had to restrict ourselves to EC2 + server-less apps)

Preparation for GSoC

I was going through the list of organizations selected for GSoC 2021 and found XWiki there. While going over the list of proposed projects, the “Amazon AMI for Wiki” got my attention as the requirements of the project align with my experience in the field of Cloud Computing. I started familiarizing myself with community guidelines and made my first forum post introducing myself. I contacted mentors assigned to the project and showed my interest. 

During the application period, I tried to understand the technical knowledge that was required for the project and made 3 pull requests according to Jira issues that my mentor raised. I tried different ways of installing XWiki that were already available and started to figure out what AWS resources would be required for XWiki to run. My mentor suggested me to use CDK as it will be easy to replicate Infrastructure as a code and also to keep track of the work being done. Then I started working on my proposal. I wrote whatever I understood to be required for the project and got it reviewed by mentors. Fortunately to my surprise, I was selected for GSoC 2021 under XWiki. 

Work Summary

For me, The most challenging part of the project was to decide on the architecture to use to run XWiki without interfering with other servers that might be running in the user's AWS account. With the help of my mentor, I finalized the system design for production-based installations on AWS and created the design wiki for the same. The next step was implementation. But before implementing the design via CDK code I started working on making AMI or EC2 instance using AWS console. @Fawad Ali helped a lot while I was getting repeatedly stuck. After that, I started working on CDK code to deploy XWiki demo/test (with a portable database (HSQLDB) and a lightweight Java container (Jetty)) instance on the user's AWS account. It was just coding the process that I did manually on the console. I also added an easy to edit config-file to change the XWiki version and AWS region according to the user's need. This installation method will require to use of AWS Command Line Interface. 

The next step was writing  CDK code for installing the production instance of XWiki, without interfering with the already deployed resources in the user's AWS account. Since we will need a VPC so I created a stack for VPC(Virtual private cloud) that will contain all of the other resources. The job zero would be to create encryption keys as they will be needed later for storing the passwords of resources. For hosting XWiki we will need to store files so for that I created a stack for AWS EFS (Elastic File System) This storage was encrypted by using previously created encryption keys and was located in the private subnet part of the VPC. For the database, I chose to use AWS fully managed Aurora DB with MySQL engine. Then I made a stack for ECS( Elastic container service) and the environment variables for connecting to the database from the ones given in XWiki container documentation and Adding the XWiki Docker image from Docker Hub. For ECS fargate service we needed a task definition that involved specifying vCPU resources available to our XWIKI container, RAM resources available to our XWIKI container, and Volumes available to our XWIKI container. The CPU and RAM specification was specified aligned to the XWiki official documentation. and for volume, It basically the connection of the XWiki container to the EFS previously made. after connecting all the resources the last step was to create the ingress layer or connection layer. I then created a stack for AWS loadbalancer which will be open to the world for outbound connectivity and for the target I added ECS fargate service to make the XWiki installation available to the world. 

Based on the same architecture I created  AWS cloudformation template for XWiki installation using AWS console. This installation method won't require AWS Command Line Interface and can be done in few clicks, by taking user-chosen version XWiki as parameter input. This was done for people with not much technical knowledge to install XWiki without installing anything on the local machine and just by using the AWS console.

Made Amazon Machine Image from EC2 instance running production XWiki (Had to do this because of constraints provided by AWS marketplace). The AMI was not published to the marketplace due to concerns regarding ownership of the project due to credit card requirements for AWS account to publish the AMI from. The AMI has been created in the XWiki SAS AWS account. The AMI has been tested and verified and the XWiki SAS can take it further as per their requirements.

Next, I made Documentation explaining the project and user guide providing a step-by-step guide to deploy XWiki by using cloudformation template and CDK code. also for a better understanding of the project, I Made videos explaining each method and step to deploy XWiki using both installation methods that is CDK code and cloudformation template to put on XWiki's you tube channel to provide better visibility.

Pull-Requests

Future Plans

I intended to improve this project and also it can act as the starting point for anyone who would be interested in contributing to the project further. A few of the improvements and new features could be the following:

  • Automating the AMI building process for every new version release of XWiki.
  • Extending infrastructure as code for supporting XWiki with PostgreSQL and MariaDB. Currently, it supports MySQL only.
  • Providing installation methods for other cloud-providing platforms, like GCP and Azure.

Remarks

I had a fun and learning experience while working under XWiki. I was afraid of asking questions initially which I overcame to some extent in the whole journey. I loved the XWiki community and I got guidance and help whenever I got stuck. i would like to thank especially @Sachin Chopra​​@Vincent Massol, @Fawad Ali, @Thomas Mortagne, @Eduard Moraru, Mr. Clément @Ludovic Dubost

.

 

Jan 19 2021

Highlights of XWiki 12.x cycle

January is here and this means we've just finished the XWiki 12.x cycle. It's time to look back at what we achieved last year, in 2020. Here are some links to visual reports of what was done in 12.x:

Dec 12 2019

Highlights of XWiki 11.x cycle

December is here and this means we're finishing the XWiki 11.x cycle. It's time to look back at what we achieved this year, in 2019. Here are some links to visual reports of what was done in 11.x:

Oct 08 2019

FOSDEM 2020 - Collaborative information and content management applications - Call for participation

The XWiki development team is very happy to announce the call for participation of the devroom "Collaborative information and content management application devroom", held during the 2020 edition of FOSDEM, on the morning of Saturday February 1st. ...

Aug 26 2019

My GSOC19 Experience

Hello there! My name is Divyansh Jain. I am a student at Mahatma Jyoti Rao Phoole University. I have been selected as an Android developer for the XWiki organization and I've finished working on their XWiki Android Authenticator application in GSOC19. ...

My GSOC journey with XWiki

Mentors:

I want to thank my mentors Shubham Jain, Neha Gupta and XWiki organisation for giving me such an opportunity and helping me out throughout my journey of GSOC. I also want to also thank Vincent Massol for helping me out in one of my dependent project.

A little bit about me

My name is Ashish Sharma, I am 23 and I am a final year student from Bharati Vidyapeeth's College of Engineering, India. I have participated in GSoC'19 program as a student in XWiki and worked on "Helm chart for XWiki".

What did I expect from GSoC 2019

This was my first Google Summer of Code and the concept to contribute for 3 months in an open-source organisation was exciting for me. I got an opportunity to work remotely on a project. I was curious to explore how an open-source organisation work. Anyway, I have expected new experience in remote internship, open source project contributing, Google Summer of Code participating and international only remote communication.

In this post I want to describe what I have finished and what is not completed yet and why.

Description

This project focoused on creating a helm chart that would deploy xwiki on Kubernetes by using helm templating. The deployment should be configurable to be scalacble, highly available and roboust.

You can find the project code at

https://github.com/xwiki-contrib/xwiki-helm and

The issue tracker can be found

https://jira.xwiki.org/projects/HELM/issues

How to use the result of my work

git clone https://github.com/xwiki-contrib/xwiki-helm
cd xwiki-helm-chart
helm dependency update
helm --debug upgrade -i --force xwiki -f ./values.yaml .

Read about prerequite

Milestones

Milestones 1:

Creating a basic helm chart

  • Understanding xwiki's docker structure.
  • Understanding and taking the decision for using the correct resource to host xwiki.
  • So I started up using the StatefulSet, clusterIP service and ingress.
  • Added values files which help in making the chart dynamically configurable.
  • Then we neded to support multiple databases and was in a need of a solution that don't require to deploy database externally, so a good standard for adding dependency for other services like database were needed.
  • So we decided to add database as a helm dependency, that helped us to templatise the databases in our application which makes deploying databases easier, we don't need to deploy the chart seprately.
  • So we added support for database mysql and postgress through dependency and also given option for user to deploy it's own external database.

Milestestone 2:

Adding support for ISTIO and other features

  • Till now we have a chart that would deploy our standalone application, now we need
  • Added support for ISTIO
  • Till now the helm were taking values from values file, we needed a way to manage our envoirnment variable that we pass to our container, and we also need to secure the sensetive variables like db password. For that I used Kubernetes resources configMaops and Secrets. These help us save and manage our variable securely and properly.
  • XWiki-helm also needed an optional feature to havdle Pod disruption, if the users have a heavy dependency on xwiki. This features make the xwiki chart user configure the minimum number of pods that should be running while pods are been destroyed due to any reason.
  • A need for shared file storage were needed, secially when serviec provider like GKE does not provide ReadWriteMany option, so to run xwiki on cluster we need different database solution like Rook, I had researched about it and also written a blog on how to set up Rook on GKE for shared file system

Milestone 3:

Adding support for clustering,HA and Unit testing

  • While moving towards HA and clustering we got to know that XWiki need shared file storage for clustering which was not available in StatefulSet, so we made a desicion to migrate to Deployment.
  • We needed to add test-cases so I added unit-testing.
  • Clustering was required to take xwiki on HA. For that we need docker to enable option to configure clustering, so first I made changes in docker project and provided option to configure JGroups.
  • Then I needed to provide the option in helm. For that I had to face another challenge to how to pass configuration files needed by JGroup from my helm chart to the container. For that I took the file in ConfigMaps and mounted the file to the xwiki pods. Also provided option to enable and disable JGroups.
  • While enabling clustering we need to know which pods are avialable to accept request and which are dead so that we can route traffic accordingly, for that I needed to configure liveliness and readiness probes, so that kubernetes always get updated with the state of the container and if the container crashes it could restart the pod.
  • XWiki uses solr and the docker it embededs solr internally, but as suggested by XWiki for performance improvement solr should be externalised, so I tried to add it as a dependency but failed due to an immature helm chart of solr, and how the xwiki configures solr, it was not possible with the current solr chart state. So for now I have exposed the basic parameter and the user has to manually setup solr and pass the url of the solr.

Jun 26 2019

Reasons to choose Kotlin

Hello there! My name is Divyansh Jain. I am a student of Mahatma Jyoti Rao Phoole University. I have been selected as the Android developer for the XWiki organization and I am working on their XWiki Android Authenticator application in GSOC19.

XWiki android Authenticator aims to integrate a wiki instance in Android accounts, mainly including the synchronization of contacts and the XWiki authenticator. By synchronizing contacts of your company on your phone, it becomes easier to communicate and collaborate with each other.

Ever since the start of my work on the XWiki Android Authenticator app, I have been constantly learning new things. In the first week, I migrated most of the XWiki Android Application code from Java to Kotlin. And on this page, I would like to share my understanding of Kotlin that I have gained so far.
 

Getting started with Kotlin

Kotlin is officially supported by Google for mobile development on Android. It was released in Android Studio 3.0 on October 2017. At first, I was a bit afraid to switch to Kotlin since I feared that the code might crash and not run properly, but as I read the documentation, I started understanding the nuances of the language which made the switch from Java to Kotlin an easier process. I started realizing the advantages of Kotlin over Java. Some of them being:

Java Interoperability

I started with migrating the whole XWiki Android Authenticator app code from Java to Kotlin. I was replacing one Java file at a time, and while migrating I saw that Kotlin worked with Java smoothly. Though it required some direct imports, there were no errors in running the app on the device.

Changed variable declaration

In Java, we declare string for instance, String str = "Hello";.

In Kotlin, we declare string for instance, val str = "Hello" or val str: String = "Hello". Here val declares a read-only property or local variable whereas var declares a mutable property or local variable.

The final keyword is default in class

In Kotlin final is a default. E.g.

             
class Button {
   fun click() = print("Click")
}

class displayToast : Button() {  // Error
   override fun click() = print("Toast Displayed") // Error
}

In the above example, class displayToast can’t inherit Button class because it is final. Moreover, it can’t override click(), because it is final in Button.

open class Button {
   open fun click() = print("Click")
   fun doubleClick() = print("Double Click")
}

class displayToast () : Button {           // Inheritance is now possible
   override fun click() = print("Toast Displayed") // Now it works
   override fun displayToast () = print("Toast Displayed") // Error
}

In order to inherit and override, we put “open” keyword that allows inheritance and overriding.

Fun keyword for defining functions

Now in Kotlin, there is a new way to define functions. E.g.

fun displayToast() { } //with no arguments inside functions

fun addDigitis (a: int, b: int) : String { }  //with arguments inside function

It is same as the Java parameterized method or empty method.

The when expression

The "switch-case" is replaced with the much more readable and flexible "when" expression: E.g.

int x = 3
when (x) {
   1 -> print("x is 1")
   2 -> print("x is 2")
   3, 4 -> print("x is 3 or 4")
   in 5..10 -> print("x is 5, 6, 7, 8, 9, or 10")
   else -> print("x is out of range")
}

It works without the argument too.

Static keywords

For declaring static methods & variables, you can put them above the class name, then you can use them by importing directly in other classes.

Null Safety

One of the biggest flaws in Java is the way it handles “null,” leading to the dreaded NulPointerException (NPE). Kotlin resolves this by distinguishing between non-null types and nullable types. Types are non-null by default, and can be made nullable by adding a safe call ‘?’. E.g.

var a: String = "abc"
a = null                // compile error

var b: String? = "xyz"
b = null                // no problem

Conclusion

So after seeing, reading and migrating the code from Java to Kotlin, in my honest opinion, I do not see any reason to not choose Kotlin over Java. For instance, we need to write less code as compared to Java, we don't have to face the dreaded ‘NPE’ error anymore, interoperability with existing Java files, smart casts while declaring variables and many more. We've given the fair amount of our time to Java, but it's time to let it go and welcome our new friend Kotlin.

Happy Reading.

Oct 19 2018

Collaborative information and content management application devroom - Call for participation

The XWiki development team is very happy to announce the call for participation of the devroom "Collaborative information and content management application devroom", held during the 2019 edition of FOSDEM, on the afternoon of Saturday 2nd February.

Please see below for the full call for participation.

Call for Participation

This is the call for participation in the FOSDEM 2019 devroom on Collaborative information and content management applications, to be held on Saturday 2nd February 2019 in Brussels, Belgium. We are looking for contributions under the form of conferences, talks or discussions covering the following main topics:

  • New projects presentation
  • Self-hosted platforms
  • Secure collaboration
  • Collaborative applications
  • Connectors & Integrations
  • Knowledge management techniques
  • Protocols for collaboration

If you have any idea of a talk fitting into the subject of the devroom, please don't hesitate to let us know about it!
We hope to provide an opportunity for everyone to meet and exchange about collaborative issues encountered while using Open Source solutions ; present the some tools used for content management and discuss about global solutions for incentivizing collaboration.

Submission process

Please submit your proposals at https://penta.fosdem.org/submission/FOSDEM19 before December 10th 2018.

If you already have a Pentabarf account (for example as a result of having submitted a proposal in the past), make sure you use it to log in and submit your proposal. Do not create a new account if you already have one. Please provide a bit of information about yourself under Person -> Description -> Abstract. When you submit your proposal (creating an "Event" in Pentabarf), make sure you choose the "Collaborative information and content management applications devroom" in the track drop-down menu. Otherwise your proposal might go unnoticed. Fill in at least a title and abstract for the proposed talk and a suggested duration. Bear in mind that a lot of the value in these meetings comes from the discussions, so please be reasonable regarding the duration of the talk.

Important dates

December 10th 2018: deadline for submission of proposals
December 16th 2018 or before: announcement of final schedule
February 2nd 2019, in the afternoon: devroom day

Recordings

The FOSDEM organizers hope to be able to live-stream and record all the talks. The recordings will be published under the same license as all FOSDEM content (CC-BY). Only presentations will be recorded, not informal discussions and whatever happens during breaks between presentations. By agreeing to present at FOSDEM, you automatically give permission to be recorded. The organizers will agree to make exceptions but only for exceptional and well-reasoned cases.

Other information

This developer room is backed up by the following projects : 

  • Cryptpad
  • Nextcloud
  • OW2
  • Tiki
  • XWiki

To get a bit more information about the devroom itself, including the initial proposal of the devroom itself, please see this page.

If you have any question about this devroom, please don't hesitate to get in touch by sending us an email to collab-devroom-manager@fosdem.org.

Thanks,
The XWiki development team

Oct 12 2018

XWiki & CryptPad hackathon in San Francisco

Between the 12th and the 20th of October, Ludovic Dubost, Anca Luca and Clément Aubin will be in San Francisco. They're looking forward to meeting contributors and friends from the area, so let them know if you want to grab a coffee and exchange ideas.

What's more is that you can join themat the hackathon hosted by Noisebridge, on Saturday, 20 October. The event will start with a presentation about CryptPad, the end-to-end encrypted realtime collaboration tool, followed by the hackathon on CryptPad and XWiki, the open source software, where you can bring your contribution to the code.

The host will be Steve Phillips, creator of CrypTag & Cypherpunks Write Code and lead developer of LeapChat. Noisebridge is a hackerspace for technical-creative projects, doocratically run by its members. They are a non-profit educational institution intended for public benefit. Located in the heart of San Francisco, their motto is: We teach, we learn, we share. 

For more details about the event read a blog article posted on the XWiki SAS blog.

Aug 23 2018

Google Summer of Code 2018 Wrap-Up

This is the eleventh year XWiki has participated in the Google Summer of Code program. XWiki had 12 mentors involved at various levels and 3 successful projects implemented. ...

Tags:
    

Get Connected