News Aggregator


Population-Based Training (PBT) Hyperparameter Tuning

Aggregated on: 2023-09-05 23:16:40

In this article, I will be talking about the population-based training hyperparameter method with an example. You can refer to my previous article to learn more about hyperparameter tuning. Hyperparameter tuning is a critical aspect of machine learning model development that involves finding the optimal combination of hyperparameters to achieve the best performance for a given dataset. Traditional grid search and random search methods are often time-consuming and inefficient, especially when dealing with complex models and large datasets. To address these challenges, population-based training (PBT) has emerged as an effective approach to hyperparameter tuning. In this article, we will delve into the concept of PBT and its advantages and provide a detailed example using the XGBoost algorithm.

View more...

Creating Custom Skills for Chatbots With Plugins

Aggregated on: 2023-09-05 22:46:40

The field of conversational AI has rapidly advanced, with Large Language Models such as ChatGPT, Claude, and BARD demonstrating impressive natural language abilities. With chatbots becoming more capable, businesses are eager to use them for customer support, document generation, market research, and more. However, out-of-the-box chatbots have limitations in terms of industry-specific knowledge, integration with business systems, and personalization of responses. The out-of-the-box chatbots can be enhanced by augmenting them with specific capabilities that perform specialized tasks that can work together to solve the problem at hand. These capabilities can be implemented as plugins that can be invoked with the help of LLMs. By adopting a plugin framework, the foundational models can be leveraged to orchestrate and invoke the necessary plugins in the right sequence. 

View more...

A PDF Framework That Solves the Pain Points of Enterprise Development

Aggregated on: 2023-09-05 22:31:40

What Is Nanhu-Print-Java? Nanhu-print-java is a PDF generation framework implemented in the Java language.  Users can prepare the JSON format business data and an XML format configure file, then call the nanhu-print-java framework API to complete the generation of a PDF file.

View more...

A NoSQL Database: MongoDB

Aggregated on: 2023-09-05 21:31:40

With the rise in data all around the world, there has been an observable and increasing interest surrounding the wave of the non-relational database, also known as ‘NoSQL. ‘ Businesses and organizations are seeking new methods to manage the flood of data and are drawn toward the alternate database management tools and systems that are different from the traditional relational database systems. Here comes MongoDB into the picture.  What Is MongoDB?  As a definition, MongoDB is an open-source database that uses a document-oriented data model and a non-structured query language. It is one of the most powerful NoSQL systems and databases around today. 

View more...

Chat to Your Requirements: My Journey Applying Generative AI (LLM) To Software Requirements

Aggregated on: 2023-09-05 21:16:40

In the digital age, large enterprises are plagued by a lack of understanding of their legacy systems and processes. Knowledge becomes isolated in silos, scattered among various teams and subject matter experts. This fragmentation contributes significantly to the growth of technical debt, a silent killer that gradually hinders the organization's agility and productivity. At Curiosity Software, we have spent the past five years creating structured requirements (through visual models) and connecting to hundreds of DevOps tools. We believe this puts us in an incredibly advantageous position when implementing Generative AI. The models and DevOps artifacts can act as the central hub of access to data flowing through an organization’s software development landscape. As part of our mission to combat the prevalent challenges of technical debt and missing knowledge, we embarked on a journey to apply Large Language Models (LLM) to software requirements and business logic. Our goal was to create a knowledge hub for an organization’s software requirements, which can be queried to uncover knowledge and pay off technical debt.

View more...

The Evolution of Bugs

Aggregated on: 2023-09-05 20:16:40

Programming, regardless of the era, has been riddled with bugs that vary in nature but often remain consistent in their basic problems. Whether we're talking about mobile, desktop, server, or different operating systems and languages, bugs have always been a constant challenge. Here's a dive into the nature of these bugs and how we can tackle them effectively. As a side note, if you like the content of this and the other posts in this series, check out my Debugging book that covers this subject. If you have friends who are learning to code, I'd appreciate a reference to my Java Basics book. If you want to get back to Java after a while, check out my Java 8 to 21 book.

View more...

One-Click Deploying EMQX MQTT Broker on Azure Using Terraform

Aggregated on: 2023-09-05 19:46:40

MQTT is a lightweight messaging protocol used in the Internet of Things (IoT) to enable communication between devices. As a popular open-source MQTT broker, EMQX provides high scalability, reliability, and security for MQTT messaging. By using Terraform, a widespread Infrastructure as Code (IaC) tool, you can automate the deployment of EMQX MQTT Broker on Azure, making it easy to set up and manage your MQTT infrastructure.

View more...

Creating a Deep vs. Shallow Copy of an Object in Java

Aggregated on: 2023-09-05 18:46:40

When working with objects in Java, there are times when you need to create a copy of an object. However, not all copies are the same. In fact, there are two main ways to make copies: deep copy and shallow copy. Let’s explore these concepts and see how they work with some easy examples. Deep Copy: What Is It? Imagine you have a collection of shapes, each with its own set of properties. A deep copy of an object means creating a completely new copy of the original object, along with all the nested objects it contains. In other words, it’s like making a photocopy of each shape, including all the details.

View more...

Battle of the Views: ClickHouse Window View vs. Live View

Aggregated on: 2023-09-05 18:46:40

ClickHouse added experimental Window Views starting from 21.12. Window Views aggregate data over a period of time and then automatically post at the end of the period. They add another exciting tool for stream processing and analytics to ClickHouse’s toolbox while expanding its collection of view tables. In general, the concepts behind Window View can also be found in other systems, such as Azure Stream Analytics, Kafka Streams, and Apache Spark, among others. Window Views follow the addition of experimental Live View tables added in 19.14.3.3. In this article, we will look at both of them to find the differences and similarities between the two. Let’s get a better view of these two different views!

View more...

How To Configure A ‘Kerberized’ Kafka Deployment For ClickHouse

Aggregated on: 2023-09-05 18:31:40

Kerberos is a secure authentication protocol for networked systems. It was developed at the Massachusetts Institute of Technology (MIT) and was first released in the 1980s. The protocol provides secure authentication of users and services, allowing them to communicate over an insecure network such as the Internet. Kerberos uses secret-key cryptography to secure communications and relies on a trusted third-party authentication server known as the Key Distribution Center (KDC). One implementation of Kerberos is Microsoft’s Active Directory, a.k.a. AD, which provides centralized authentication and authorization. Active Directory uses Kerberos to provide single sign-on capabilities for Windows-based computers, allowing users to access network resources with a single set of credentials. In addition to managing user and computer accounts, Active Directory also includes features such as group policy, trust relationships, and the ability to integrate with other directory services such as LDAP and NIS. Also, AD allows users to integrate the authentication of external services like Kafka or ClickHouse. 

View more...

7 Essential Software Quality Metrics for Project Success

Aggregated on: 2023-09-05 18:31:40

Let's talk about something crucial in software projects—software quality metrics. These metrics can help us ensure our projects are successful and meet the needs of the people using them. In this article, our primary purpose is to provide you with a comprehensive guide to the essential software quality metrics that can significantly contribute to the success of your project. We want you to walk away with a solid understanding of these metrics so you can make well-informed decisions and ensure the best outcomes for your projects.

View more...

Getting Started With the IBM App Connect Public API

Aggregated on: 2023-09-05 18:31:40

If you are looking to create App Connect resources programmatically or provide your own monitoring and administration capabilities, this article offers an introduction to using the public API, with a worked example using Bash for those looking to get started. With the App Connect public API, you can programmatically perform the following tasks:

View more...

What Is Langchain And Large Language Models?

Aggregated on: 2023-09-05 17:46:40

If you're a developer or simply someone passionate about technology, you've likely encountered AI tools such as ChatGPT. These utilities are powered by advanced large language models (LLMs). Interested in taking it up a notch by crafting your own LLM-based applications? If so, LangChain is the platform for you. Let's keep everything aside and understand about LLMs first. Then, we can go over LangChain with a simple tutorial. Sounds interesting enough? Let's get going.

View more...

The 10 Laws of Testing

Aggregated on: 2023-09-05 17:31:40

Words we as developers should live by: “Quality comes not from inspection, but from the improvement of the production process.” — W. Edwards Deming Axiom: “Tests are code.” Too many organizations treat anything that isn’t coded as disposable. It is obvious that testing is indispensable, yet time and time again we find teams who treat test automation and related material as second-class citizens. The tests are documents of user behaviour, inextricably linked to the requirements produced by the Product organization and connected at the virtual hip to the code used to create the feature. If it provides value, it should be versioned, maintained, cared for, and respected as though it were a core feature of the product itself. This should include test case specifications, design and technical documentation, and bug reports.

View more...

Creating a Custom Starter With Spring Boot 3

Aggregated on: 2023-09-05 17:31:40

Starters are an integral part of the Spring Boot application. In addition to dependency versioning, they provide the ability to describe the configuration for a particular functionality. They gained their popularity due to the development of microservice architecture. When we have dozens or hundreds of services within one application, some of the functionality is not duplicated. For example, we need to connect the cache, database, logging etc. In order not to duplicate the same configuration every time, we simply add the starter as a dependency to our microservice, and Spring recognizes all our configurations. We will try to create our own simple starter on Spring Boot 3. There have been some changes in it compared to the second version. Our starter will have the simplest functionality, namely to output a message to the console when our application starts. We create a project through spring initializer, which we will call according to the custom-spring-boot-starter convention

View more...

Component Library With Lerna Monorepo, Vite, and Storybook

Aggregated on: 2023-09-05 17:01:40

Building components and reusing them across different packages led me to conclude that it is necessary to organize the correct approach for the content of these projects in a single structure. Building tools should be the same, including testing environment, lint rules, and efficient resource allocation for component libraries. I was looking for tools that could bring me efficient and effective ways to build robust, powerful combinations. As a result, a formidable trio emerged. In this article, we will create several packages with all those tools.

View more...

AI in Edge Computing: Implementing Algorithms to Enhance Real-Time

Aggregated on: 2023-09-05 17:01:40

Edge computing is a transformative approach that places computation near data sources such as IoT devices, sensors, and network switches. This localized processing paradigm has gained traction in various fields, and its integration with Artificial Intelligence (AI) is revolutionizing real-time decision-making. Below, we explore how AI and edge computing are converging to enable faster and more efficient processing, specifically in the context of Internet of Things (IoT) devices. The Emergence of AI in Edge Computing The traditional model of processing data in centralized cloud servers has been challenged by the explosion of data generated by IoT devices. Transferring enormous volumes of data to a centralized location for processing increases latency and raises concerns about bandwidth usage and security. AI in edge computing emerged as a solution to these challenges, bringing intelligent decision-making directly to the source of data generation.

View more...

How To Learn Software Development Without a Degree

Aggregated on: 2023-09-05 16:31:40

Software development has become one of the most sought-after skills. The demand for skilled software developers continues to grow, and many aspiring individuals are eager to learn this skill even if they don't have a formal degree in computer science or a related field. If you are passionate about software development and wondering how to get started without a degree, this article is here to guide you. We will explore various strategies and resources that can help you embark on a successful journey into the world of software development. Understand the Basics Before diving into the complexities of software development, it is essential to understand the fundamentals. Familiarize yourself with programming languages like Python, Java, JavaScript, or Ruby. Online tutorials and interactive coding platforms can be incredibly beneficial in this regard. You can learn about data structures, algorithms, and basic coding principles to build a solid foundation.

View more...

The Role of JavaScript in Front-End and Back-End Development

Aggregated on: 2023-09-05 13:46:40

JavaScript: the versatile programming language that's at the heart of modern web development. When it comes to both front-end and back-end development, JavaScript plays a pivotal role in shaping the digital landscape. In this article, we'll delve into the significance of JavaScript in these two spheres while also drawing comparisons to another powerhouse, PHP. Let's embark on a journey through the world of web development and explore how JavaScript has redefined the way we create and experience digital content. Introduction to JavaScript and its Evolution JavaScript, often abbreviated as JS, was introduced in the mid-1990s as a scripting language for web pages. Over the years, it has undergone significant evolution, emerging as a robust and multifunctional language that not only enhances user interfaces but also powers server-side applications.

View more...

Defect vs. Bug – The Key Differences

Aggregated on: 2023-09-05 13:46:40

There would be no software engineer, QA engineer, or IT professional in the industry who has not heard the terms "bug" and "defect." However, how many of them know the difference between a bug and a defect? You might say that both terms mean almost the same thing. But that would be incorrect. In this article, we are going to have a look at what defects and bugs are, how defects and bugs differ from each other, and how they affect the software development process. What Is a Bug? A bug is a coding error a developer makes during the coding phase of the software application. A bug can be a syntax error or a logical error in the code that will give you incorrect results or break the application. A bug is a term usually used during the software development life cycle. For example, when a software tester detects an issue in the application, he/she will report it to the developer as a bug. 

View more...

Develop XR With Oracle, Ep. 6: AI Summarizer + Generator

Aggregated on: 2023-09-05 12:46:40

In this sixth article of the series, we describe and provide the full source for a mixed reality app that takes any visual and audio input from a user's surroundings, processes it with various AI, and returns output of summaries, results of generative AI, etc. This is the sixth piece in a series on developing XR applications and experiences using Oracle and focuses on visual and audio input from a user's surroundings, processing it with various AI, and returning output of summaries, results of generative AI, etc. Find the links to the first five articles below: 

View more...

Navigation the Testing Challenges of Global Migration to ISO 20022

Aggregated on: 2023-09-05 04:01:40

In today's fast-paced financial landscape, where seamless data exchange is crucial for operational efficiency, the adoption of ISO 20022 has emerged as a global standard for financial messaging. While the advantages of ISO 20022 are widely recognized, the migration process presents numerous challenges for organizations, particularly when it comes to testing. In this article, we will explore how the migration to ISO 20022 globally affects testing and the strategies organizations can employ to overcome these hurdles. 

View more...

Topology Spread Constraints for Increased Cluster Availability and Efficiency

Aggregated on: 2023-09-05 03:31:40

Running a large-scale application serving customers across multiple regions can be challenging. You can use different methods to control the Kubernetes scheduler, but if you long for high availability and efficiency across different failure domains, it’s time to explore topology spread constraints. Optimizing performance and availability can be tricky even when your systems don’t comprise numerous data centers across different countries or continents. Node Selector, Node Affinity, Pod Affinity, and Anti-affinity can help you tame the Kubernetes scheduler, but they may be insufficient for setups with multiple failure domains like regions, zones, and nodes. 

View more...

This Is How You Automate Creation of Recurring Tickets in Jira

Aggregated on: 2023-09-05 03:16:40

Jira is a pretty powerful tool when it comes to automating recurring processes. You can easily automate Jira ticket creation, for example. The whole process probably wouldn’t take longer than ten minutes to set up, especially when you know what you are doing and you follow Jira best practices. Let’s say you need to make a monthly report on the performance of your blog. You’ll need to check total traffic, conversions, time on page, etc. Jira allows you to set up certain automation rules that will automatically create a ticket on the date you’ve set up.  

View more...

Exploring the Basics of EMQX MQTT Broker Clustering: An Introduction

Aggregated on: 2023-09-04 18:31:40

Welcome to the first post in our new series exploring the world of MQTT broker clustering. If you're involved in the IoT (Internet of Things) space or have embarked on any journey involving real-time data transfer, you've probably come across MQTT (Message Queuing Telemetry Transport). MQTT is a lightweight, publish-subscribe network protocol that transports messages between devices, often known as the backbone for IoT.

View more...

Taking the Fear Out of Migrations

Aggregated on: 2023-09-04 17:01:40

Over the last 18 months at incident.io, we’ve done a lot of migrations. Often, a new feature requires a change to our existing data model. For us to be successful, it’s important that we can seamlessly transition from the old world to the new as quickly as we can.  There are few things in software where I’d advocate a ‘one true way,’ but the closest I come is probably migrations. There’s a playbook that we follow to give us the best odds of a smooth switchover: Make the new data models. Write to both the old and the new model. Start reading from the new model. Drop the old data model. A Worked Example Last year, we introduced customizable incident statuses. Before that, our statuses were hard coded as strings (investigating, fixing, monitoring, closed). There was a database column status which, for each incident, contained one of those four strings. We wanted to move to a world where customers could choose their own statuses, both renaming them and choosing how many they wanted. So how did we do this? 1. Make the New Data Models We’ve got to agree on the target data model: what the new world will look like. We need a new table to represent the statuses configured against each organization: incident_statuses, and a column on our incidents table to store which status it’s currently in: incident_status_id.

View more...

Part 4: Validating CDC Messages With Schemaverse

Aggregated on: 2023-09-04 16:16:40

This is part four of a series of blog posts on building a modern event-driven system using Memphis.dev. In the previous two blog posts (part 2 and part 3), we described how to implement a change data capture (CDC) pipeline for MongoDB using Debezium Server and Memphis.dev.

View more...

The Ultimate Guide to API vs. SDK: What’s the Difference and How To Use Them?

Aggregated on: 2023-09-04 15:31:40

First, let’s figure out the terms we are using and why API vs SDK are even paired together. What Is an API? API (an acronym for Application Programming Interface) is an interface that enables intercommunication between two applications. It includes a standardized set of rules that define how this interaction can undergo, i.e., what kind of information to exchange, what actions to carry out, etc.

View more...

Single-Tenant vs. Multi-Tenant Architecture: Breaking Down the Key Differences

Aggregated on: 2023-09-04 14:31:40

SaaS applications and services are at the core of today’s businesses, and a quick glance at the market indicates that this trend isn’t going to stop anytime soon. Gartner forecasts that SaaS spending will reach $197 billion in 2023, up 17.9% from last year, with another 17.7% growth expected in 2024 to $232 billion. Furthermore, the average number of SaaS applications used by organizations worldwide reached 130 in 2022, up from 80 in 2020 and just 16 in 2017. As the number of SaaS products and vendors continues to grow, one of the most important decisions they need to make is how to seamlessly and successfully deliver their software to their customer base.

View more...

Introduction to the Graph Data Structure

Aggregated on: 2023-09-04 13:31:40

Graph data structure is a fundamental concept in computer science and mathematics that models relationships between objects. A graph consists of vertices or nodes connected by edges, which represent the relationships between the vertices. Graphs are used in various applications, such as network design, social network analysis, and recommendation systems. In this article, we will explore the graph data structure, its features, and its usage in programming. Features of Graph Data Structure The graph data structure has some essential features that make it an excellent choice for modeling relationships between objects. Some of the key features of the graph data structure are:

View more...

What’s a Merge Queue and Why Use It?

Aggregated on: 2023-09-03 12:01:39

That’s a good question, isn’t it? Indeed, if it was only a little-known term a few months ago, merge queues are more and more talked about. Between announcements from leaders like GitHub and real technical solutions, merge queues are increasingly adopted by software development teams. You will therefore dive into this topic to understand what a merge queue is, in what situations they are useful but also how they work in practice.

View more...

Introduction to the Map Data Structure

Aggregated on: 2023-09-02 17:31:39

Maps are a fundamental data structure used in computer science to store and organize data in an efficient way. Maps store a collection of key-value pairs, which allows you to retrieve a value based on its associated key. This makes them particularly useful when you have large datasets and need to quickly access specific pieces of information. The implementation of maps varies depending on the programming language and are frequently used in Python, Java, and JavaScript. A map is implemented as a dictionary in Python, whereas it is implemented as a hashmap in Java. Despite these variations, the fundamental ideas behind maps are the same.

View more...

12 Agile Principles for Successful Agile Development Practices

Aggregated on: 2023-09-02 17:16:39

Injecting Agile software development principles into your SDLC helps unlock greater adaptability, agility, performance, and value for all stakeholders — customers, organizations, and investors.  As per a report, 72% of people are very satisfied or somewhat satisfied with adopting Agile development practices. But the remaining are not that happy with the outcomes. 42% quote inadequate leadership participation as the barrier to successful agile delivery. Conflict with the existing organizational culture, resistance to change, heterogeneous SDLC practices, and insufficient training and experience are some other challenges staring into the eyes of those who aspire to go agile. 

View more...

When Is the Right Time to Scale Your Digital Product?

Aggregated on: 2023-09-02 16:46:38

Your digital product is thriving, and you're witnessing the fruits of your effort. But now, you face a critical decision: When is the right time to scale? Scaling can be both exhilarating and nerve-wracking, as it can unlock untapped potential and unprecedented growth. However, leaping too soon or hesitating too long can result in missed opportunities and a halt in progress.  This article will guide you through the maze of uncertainty with practical insights and expert advice. You will learn to spot the telltale signs that scream "Go big!" and how to overcome the obstacles that may stand in your way. Prepare to take your digital product to soaring new heights — the perfect time to scale is now!

View more...

Exploring the World of Artificial Intelligence: A Beginner’s Guide

Aggregated on: 2023-09-01 22:16:38

“I believe that at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted” — Alan Turing, 1947 (Father of Computer Science and Artificial Intelligence)

View more...

Architecting TDD for Services

Aggregated on: 2023-09-01 22:16:38

Beyond Unit Testing Test-driven development (TDD) is a well-regarded technique for an improved development process, whether developing new code or fixing bugs. First, write a test that fails, then get it to work minimally, then get it to work well; rinse and repeat. The process keeps the focus on value-added work and leverages the test process as a challenge to improving the design being tested rather than only verifying its behavior. This, in turn, also improves the quality of your tests, which become a more valued part of the overall process rather than a grudgingly necessary afterthought. The common discourse on TDD revolves around testing relatively small, in-process units, often just a single class. That works great, but what about the larger 'deliverable' units? When writing a microservice, it's the services that are of primary concern, while the various smaller implementation constructs are simply enablers for that goal. Testing of services is often thought of as outside the scope of a developer working within a single codebase. Such tests are often managed separately, perhaps by a separate team, using different tools and languages. This often makes such tests opaque and of lower quality and adds inefficiencies by requiring a commit/deploy as well as coordination with a separate team.

View more...

Generative AI: Innovating Ethically and Creatively for Seamless Data Transfer

Aggregated on: 2023-09-01 21:46:38

Generative AI refers to a category of artificial intelligence techniques that involve generating new data, such as images, text, audio, and more, based on patterns learned from existing data. Generative models like Generative Adversarial Networks (GANs) and Variational Auto-Encoders (VAEs) have demonstrated remarkable capabilities in generating realistic and diverse data for various purposes, including data collection. Leverage Generative AI for Data Collection Data Augmentation Generative models can create new samples that closely resemble your existing data. By incorporating these generated samples into your training data, you can improve your model's performance and resilience, particularly in tasks such as image classification and object detection.

View more...

Janus vs. MediaSoup: The Ultimate Guide To Choosing Your WebRTC Server

Aggregated on: 2023-09-01 21:46:38

When building real-time multimedia applications, the choice of server technology is pivotal. Two big players in this space are Janus and MediaSoup, both enabling WebRTC capabilities but doing so in distinctly different ways. This comprehensive guide aims to provide a deep dive into the architecture, code examples, and key differentiators of each, helping you make an informed choice for your next project. The Role of a WebRTC Server Before diving into the specifics, let’s clarify what a WebRTC server does. Acting as the middleman in real-time web applications, a WebRTC server manages a plethora of tasks like signaling, NAT traversal, and media encoding/decoding. The choice of server can significantly affect the performance, scalability, and flexibility of your application.

View more...

LLM-Powered OLAP: The Ten Cent Experience With Apache Doris

Aggregated on: 2023-09-01 21:31:38

Six months ago, I wrote about why we replaced ClickHouse with Apache Doris as an OLAP engine for our data management system. Back then, we were struggling with the auto-generation of SQL statements. As days pass, we have made progress big enough to be referenced for you (I think), so here I am again.  We have adopted Large Language Models (LLM) to empower our Doris-based OLAP services.

View more...

Data Tiering in SAP HANA Cloud-Native Storage Extension

Aggregated on: 2023-09-01 21:16:38

In HANA Cloud, data can be stored in three categories. HOT, WARM, and COLD. HOT data is the data that resides in memory and is expensive. WARM data is the data that resides in the disk COLD data is the data that resides in a Data Lake. In this article, we will explain how we can move data to a disk and what the steps are that need to be performed to be able to move data into the DISK.

View more...

Emitting Protocol Buffers Using CockroachDB CDC Queries

Aggregated on: 2023-09-01 20:16:38

Previous Articles on CockroachDB CDC Using CockroachDB CDC with Apache Pulsar Using CockroachDB CDC with Azure Event Hubs Using CockroachDB CDC with Confluent Cloud Kafka and Schema Registry SaaS Galore: Integrating CockroachDB With Confluent Kafka, Fivetran, and Snowflake CockroachDB CDC Using Minio as Cloud Storage Sink - Part 3 CockroachDB CDC using Hadoop Ozone S3 Gateway as cloud storage sink Motivation Protocol Buffers are language-neutral, platform-neutral extensible mechanisms for serializing structured data. It's a common choice for platforms needing to pass messages between systems. CockroachDB is a distributed SQL database built on a transactional and strongly-consistent key-value store. It scales horizontally; survives disk, machine, rack, and even datacenter failures with minimal latency disruption and no manual intervention; supports strongly consistent ACID transactions; and provides a familiar SQL API for structuring, manipulating, and querying data. There is no official support for Protocol Buffers in CockroachDB Changefeeds, even though we use Protocol Buffers extensively in the code. A recent customer conversation led to this experiment where I'm going to use several recent features to demonstrate the ability to serialize CockroachDB rows to proto and emit via CDC Queries. This is the first time we're looking at CDC Queries. This is a new flexible way to express CockroachDB streams.

View more...

The Power of AI: Why Web Developers Still Reign Supreme

Aggregated on: 2023-09-01 20:16:38

Artificial Intelligence (AI) has emerged as a powerful tool in various fields, and web development is no exception. With its ability to analyze data, automate processes, and enhance user experiences, AI has revolutionized the way websites are created and maintained.  In this article, we will explore the benefits of incorporating AI in web development, the popular AI technologies used by web developers, and how AI enhances the user experience on websites.

View more...

An Approach To Construct Curated Processed Dataset for Data Analytics

Aggregated on: 2023-09-01 19:46:38

Organizations in today’s world have increased focus on data as a tool to perform various kinds of profound analysis to extract vital outcomes, which will help organizations make the right decision to define future strategies. For the success of an organization in today’s world, having the right data processing strategy on all the organization’s data to harness and extract different kinds of perspectives and information is becoming paramount to the success of the organization.

View more...

Quix Streams: Stream Processing With Kafka and Python

Aggregated on: 2023-09-01 19:31:38

Over 100,000 organizations use Apache Kafka for data streaming. However, there is a problem: The broad ecosystem lacks a mature client framework and managed cloud service for Python data engineers. Quix Streams is a new technology on the market trying to close this gap. This blog post discusses this Python library, its place in the Kafka ecosystem, and when to use it instead of Apache Flink or other Python- or SQL-based substitutes. Why Python and Apache Kafka Together? Python is a high-level, general-purpose programming language. It has many use cases for scripting and development. But there is one fundamental purpose for its success: Data engineers and data scientists use Python. Period.

View more...

How To Measure Developer Productivity

Aggregated on: 2023-09-01 19:16:38

Three primary methods for measuring team productivity are the SPACE framework, DORA metrics, and Goals/Signals/Metrics (GSM). The SPACE Framework for Team Productivity In a recent research paper by Nicole Forsgren and her colleagues, “The SPACE of Developer Productivity" (2021.), the authors defined a framework as a systematic approach to measuring, understanding, and optimizing engineering productivity. It encourages leaders to take a comprehensive approach to productivity, communicating measurements with one another and connecting them to team objectives. The five aspects are used to categorize engineering productivity, called the Space Framework.

View more...

Programming With AI

Aggregated on: 2023-09-01 18:31:38

I recently discussed how we use Co-Pilot and ChatGPT for programming with some of my senior colleagues. We discussed our experiences, how and when it helps, and what to expect from it in the future. In this article, I will shortly write about what I imagine the future of programming with AI will be. This is not about what AI will do in programming in the future. I have no idea about that, and based on my mood, I either look forward amazed or in fear. This article is more about how we, programmers, will do our work in the future.

View more...

How To Build an AI/ML Feature Store With ScyllaDB NoSQL

Aggregated on: 2023-09-01 18:16:38

Machine learning (ML) feature stores have been attracting attention and usage for business-critical applications ever since Uber introduced the concept with Michelangelo in 2017. In this blog post, we will delve into the fundamentals of ML feature stores and explore why and how ScyllaDB can be a critical part of your feature store architecture. In order to understand what feature stores are, it’s important first to understand what features are.

View more...

Are You an Efficient Developer? Then AI Is After Your Job

Aggregated on: 2023-09-01 17:31:38

In my previous post, I wrote about finding your competitive advantage as a software developer in a world of encroaching AI. I believe there are a few questions more relevant in the coming decade, so I like to elaborate on the topic in this and my next postings. You can only maintain such a competitive advantage if you focus on tasks that humans by their very nature can do better than machines. There are two necessary properties of software development that illustrate and coincide with the relative strengths and weaknesses of humans versus computers They are efficiency and effectiveness, and will be the focus of today’s post. I give credit to Uwe Friedrichsen for pointing out this distinction in a recent blog.

View more...

How To Check Office Files for Macros Using Java

Aggregated on: 2023-09-01 16:46:38

The Microsoft OpenXML files we use on a day-to-day basis are conveniently designed to be accessed and manipulated programmatically. We can jump into any OpenXML file structure in a variety of capacities (usually via specialized programming libraries or APIs) to easily manipulate objects within a document and/or retrieve important contents from its various sections.   The flexibility afforded by Office document formats is, to an even greater extent, facilitated by macros. Using the Visual Basic for Applications (VBA) programming language - a specially designed version of the Visual Basic (VB) language - we can add a myriad of dynamic elements to our Office documents and allow our files to seamlessly connect with other applications in our system.

View more...

Revolutionizing Debugging: Exploring the Test Replay in Cypress

Aggregated on: 2023-09-01 13:46:38

Founder of Cypress Brian Mann recently unveiled Test Replay, an innovative new tool that will revolutionize debugging in Cypress. The main points from the webinar are summarised in this post; if you're keen to learn more, you can access the complete recording on YouTube. Test Replay feature that the Cypress team has been working on to address debugging challenges in Continuous Integration (CI) environments. Test Replay is a powerful capability that allows you to replay and analyze test runs, making it easier to debug issues that may arise during CI testing. This is particularly beneficial because debugging in CI environments can be more challenging due to limited visibility and reproducibility.

View more...